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ABSTRACT 

This evaluation report is the third in a series of 
reports that are part of a 4-year national effort designed to 
describe the types of Even Start projects that have been funded, the 
services provided, the collaboration efforts undertaken, and the 
obstacles to program implementation that have been encountered. The 
current report provides information about the first 2 cohorts of Even 
Start projects, 76 that began in 1989 and 47 that began in 1990. The 
first six sections of the report describe the background, design, and 
characteristics of the Even Start program, its participants, and its 
core services. Section seven examines approaches to assessing the 
short-term effects of the Even Start program on the participants. 
Sections 8 through 11 discuss in detail the effects of the program on 
children, parent literacy, parenting skills, and families. Section 12 
reviews the cost of the program, while section 13 summarizes the 
entire report . (Contains 69 references .) (MDM) 
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Preface 



The National Evaluation of the Even Start Family Literacy Program is a four-year national 
effort designed to describe the types of Even Start projects that have been funded, the 
services provided, the collaborative efforts undertaken, and the obstacles to program 
implementation that have been encountered. The evaluation also describes the families 
participating in Even Start, the services they receive, and the effects of Even Start 
participation on children's school readiness; parent's literacy, parenting, and personal 
skills; and family stability and resources. Finally, the evaluation provides assistance to 
Even Start projects to conduct locally-designed evaluations, and to prepare and submit 
applications to the Department of Education for entry into the National Diffusion Network. 

This is the third report from the National Even Start Evaluation, It provides information 
about the first two cohorts of Even Start project (76 projects first funded in the fall of 
1989 and 47 projects first funded in the fall of 1990). Data for this report reflect the 
operations and effectiveness of Even Start projects as they were implemented in the 
1989-90, 1990-91, and 1991-92 school years. 

The first six sections of this report provide descriptive data which update material 
contained in earlier reports. The latter sections of the report contain new information 
about the costs and effectiveness of Even Start. 
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Section One 

Program Background 



Until recently, the Nation's literacy problems have been addressed through a dual system 
of public and private sector programs: (1) remediation programs for adults in the form of 
adult education or workplace literacy programs, and (2) prevention programs for children 
through early childhood education efforts such as Head Start (Barbara Bush Foundation, 
1989). The seeds of a new approach were sown in the late 1970s anH early 1980s 
when many of the first "family literacy" programs were planned and implemented. 
Drawing on the experiences of existing early intervention and adult literacy programs, 
family literacy programs are based on the beliefs that children's early learning is greatly 
influenced by their parents, that parents must develop and value their own literacy skills 
in order to support their children's educational success, and that parents are their 
children's first and best teachers. 

In the late 1980s this new approach emerged in full force as family literacy programs 
proliferated under a very wide range of sponsors including state governments (e.g., 
Kentucky's PACE program), school districts (e.g., the Marin, California Library Family 
Literacy Program), private foundations (e.g., the Kenan Charitable Trust's Family Literacy 
Program), private corporations (e.g., Stride Rite's Intergenerational Day Care program), 
and universities (e.g., El Paso State College's Family Intergenerational English Literacy 
Program). The movement attained national status in 1 989 when the federal government 
instituted its family literacy centerpiece, the Even Start program. 



The Even Start Family Literacy Program 

The Even Start Family Literacy Program was authorized by the Elementary and Secondary 
Education Act of 1 965 as amended by the Hawkins-Stafford Elementary and Secondary 
School Improvement Amendments of 1988, Part B of Chapter 1 of Title I {P.L. 100-297). 
In 1991, Congress passed the National Literacy Act (P.L. 102-73) which amended the 
Even Start program. According to the law, the Even Start program is intended to: 

...improve the educational opportunities of the Nation's children and adults by 
integrating early childhood education and adult education for parents into a 
unified program.... The program shall be implemented through cooperative 
projects that build on existing community resources to create a new range of 
services. (P.L. 100-297, Sec. 1051). 

To be eligible for Even Start, a family must have an adult who is eligible to participate in 
an adult education program under the Adult Education Act, and who is a parent of a child 
less than eight years of age who lives in a Chapter 1 elementary school attendance area. 
Even Start projects must provide participating families with an integrated program of early 
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childhood education, adult basic skills training, and parenting education. The program's 
design is based on the notion that these components build on each other and that families 
need to receive all three services, not just one or two, in order to effect lasting change 
and improve children's school success. As a "family-focused " rather than parent- or child- 
focused program, Even Start has three interrelated goals: 

• to help parents become full partners in the education of their children, 

• to assist children in reaching their full potential as learners, and 

• to provide literacy training for their parents. 

To achieve these goals, Even Start began as a demonstration program administered by 
the U.S. Department of Education (ED) that provided school districts with four-year 
discretionary grants for family literacy projects. In 1992, the program, while remaining 
a competitive discretionary grant program, became primarily administered by the states, 
although two small set-asides remain for direct Federal grants for Migrant Education 
projects and grants to Indian tribes and tribal organizations. According to the Even Start 
statute, when the program is funded for $50 million or more per year, it must be 
administered at the state level. Each state's share of Even Start funds is based on its 
proportion of Chapter 1 Basic Grants funds. States hold grant competitions and make 
subgrant awards. The statute specifies that each Even Start subgrantee must receive a 
minimum of $75,000 per year. Exhibit 1.1 summarizes Even Start's funding history and 
Exhibit 1 .2 shows the location of Even Start projects. 





Exhibit 1.1 
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Even Start Funding History 




Fiscal Year 


|| Federal Funding 


Number of Projects 


1989 


$14,820,000 


76 


1990 


$24,201 ,000 


123 


1991 


$49,770,000 


234 


1992 


$70,000,000 


340 (est.) 


1993 


$89,123,000 


440 (est.) 


1 994 request 


$110,000,000 


539 (est.) 


Exhibit reads: In fiscal year 1991, $49.8 million of federal funding was used to provide Even Start 
grants to 234 local projects. 


Source: U.S. Department of Education, Office of Policy and Planning. 
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Exhibit 1.2 

Location of Even Start Projects 
(Projects Funded in 1969, 1990, or 1991) 




Source: U.S. Department of Education, Office of Policy and Planning, April 1993. 



Mandate for the Evaluation 

Section 1058 of the Even Start legislation requires an independent national evaluation of 
the projects funded under Even Start. This section reads as follows: 

fa) Independent Annual Evaluation . The Secretary shall provide for the annual 
independent evaluation of programs under this part to determine their effectiveness 
in providing: 

(1) services to special populations; 
12) adult education services; 

(3) parent training; 

(4) home-based programs involving parents and children; 

(5) coordination with related programs; and 

(6) training of related personnel in appropriate skill areas. 

Even Start 1-3 Program Background 
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(b) Criteria . 



(HEach evaluation shall be conducted by individuals not directly involved in the 
administration of the program or project operated under this part Such independent 
evaluators and the program administrators shall jointly develop evaluation criteria 
which provide for appropriate analysis of the factors under subsection (a). When 
possible, each evaluation shall include comparisons with appropriate control groups. 

(2) In order to determine a program 's effectiveness in achieving its stated goals, each 
evaluation shall contain objective measures of such goals and, whenever feasible, 
shall obtain the specific views of program participants about such programs. 

(3) Reoort to Congress and Dissemination. The Secretary shall prepare and submit 
to the Congress a review and summary of the results of such evaluations not later 
than September 30, 1 993. The annual evaluations shall be submitted to the National 
Diffusion Network for consideration for possible dissemination. 



In January 1990, the Office of Policy and Planning in the U.S. Department of Education 
{ED) awarded a contract to Abt Associates Inc., with a subcontract to RMC Research 
Corporation, for an evaluation of the Even Start program. The evaluation, which runs 
from 1990 through 1993, requires the design and implementation of a four-part 
evaluation and includes annual reports to be delivered to ED as well as a final report to 
Congress. 

The evaluation assists ED and the general public in two main ways. First, it enables ED 
to fulfill the legislative requirement for an evaluation of the Even Start program. Second, 
it provides information needed for program improvement and administration by the 
Department of Education and States. Third, it adds to the knowledge base on the effects 
of family literacy programs by investigating the relationships between program processes 
and outcomes. 
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Section Two 

Program Design 



The Even Start legislation contains language setting forth the major elements that must 
be the basis of each Even Start local project. However, the legislation allows grantees 
great flexibility in devising projects to meet local needs. Even Start encourages local staff 
to draw on available program models end to collaborate with existing service providers 
to create projects that are tailored to the needs of local families. Because of this, Even 
Start can be regarded as a "family literacy laboratory" in which many different strategies 
are being tried. Early evidence reveals that Even Start includes a fairly complete 
representation of the various adult education, parenting education, and early childhood 
education programs that exist in'the nation today. 

Exhibit 2.1 presents a conceptual model depicting the types of activities conducted by 
Even Start projects and the causal chain anticipated as a result of those activities- The 
projects are characterized as having a set of program inputs which influence the creation 
of program processes, which in turn lead to several sets of outcomes for parents and 
children. At each level (inputs, processes and outcomes), a set of contextual variables 
act as mediators. Examples of measurable indicators are provided for each major set of 
variables shown in the model. 

The model shows that variation in local projects can be described along many dimensions. 
These include the collaboration strategy used by the project in deciding what services to 
provide directly and what services to provide through referrals; the characteristics of 
target children and adults {e.g., age of child, family language); the extent to which 
services for families are integrated (e.g., activities in parent education reinforce learning 
in adult education); the use of an existing educational model and materials for delivering 
early childhood and adult basic education services; strategies for recruiting and retaining 
program participants; the role that parents play in the project; and staff development 
activities. 

Further, many Even Start projects use case managers, parent liaisons, or family advocates 
as key staff in the provision of coordinated services. Case managers conduct needs 
assessments and have ongoing contact with a number of families at centers and through 
home visits. They are responsible for the direct provision of some services as well as for 
ensuring that participating families take advantage of other services. 

Three core Even Start services are required in the legislation: 

Early childhood education: services to meet the early education needs of children 
from birth through seven years of age, designed to enhance development and 
prepare children for success in school. 
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Exhibit 2.1 
Even Start Conceptual Model 
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Exhibit 2.1 
(continued) 




Literacy 
111 Behaviors 



• Shared literacy 
events with children 

• Increased use of 
literacy materials 

• Literacy resources 
in home 



I Parenting Behavior 
and Skills 




• Positive parent-child 
relationships 

• Home environment to 
foster child development |i 

• Positive expectations 
for child 

• Adequate supervision 



1:1 



-■- ri i i i r uum i n ■• 

Education Skills 1 
and Expectations 



« Increased basic skills/ . 
English language ability 

• Higher educational 
attainment 

• Improved job skills and 
employment status 



mm 



Personal Skills 



Increased self-esteem 

• Increased self-efficacy 

• Increased personal 
well-being 



Community 
Involvement 



• Increased Involvement 
in school 

• Aocess to social 
services 



••••• 





Even Start 2-3 \ j Program Design 




Adult education: services that develop the basic educational and literacy skills of 
the adult including adult bpsic education (ABE), adult secondary education (ASE), 
English as a second language (ESL), or preparation to attain a General Education 
Development (GED) certificate. 

Parent education: services for parents designed to enhance parent-child 
relationships and help parents understand and support their child's growth and 
development. 

It is expected and required by ED regulations that each family will participate in all three 
core services. Projects are free to choose the specific instructional strategies used in 
each of these core service areas. In addition, Even Start projects can decide to focus 
educational activities for children on a narrower age span than the birth through seven 
range that is targeted by the legislation. Finally, Even Start projects are required to 
provide some core services to parents and children in joint sessions and to provide home- 
based services. 

Core services may be provided by staff funded through Even Start or by staff at 
cooperating agencies. For example, early childhood education may be provided by a local 
Head Start project and adult education classes might take place at local community 
colleges. 

In addition to core services. Even Start projects typically provide a range of "support" 
services, many of which are designed to enable families to participate in core program 
activities. Examples of support services are transportation, child care, and referrals for 
employment. The Even Start legislation requires that support services be obtained from 
existing providers whenever possible, to ensure that Even Start projects avoid duplication 
of services. 

It is hypothesized that the Even Start program has the potential to benefit families in 
several domains. Desired outcomes for parents include positive effects in three areas 
linked to the Even Start legislation: literacy behaviors (e.g., shared literacy events with 
children, increased reading and writing activities in the home), parenting behavior and 
skills (e.g., positive parent-child relationships, positive expectations for child), and 
educational skills (e.g., improved reading and English language ability, higher educational 
attainment). In addition, goals for parents participating in Even Start might include 
growth in personal skills (e.g., increased self-efficacy) and community involvement (e.g., 
increased involvement in schools). 

It is also expected that Even Start will have a positive impact on children's school 
readiness and school achievement. School readiness variables include age-appropriate 
cognitive, language, and social skills. Once children enter school, outcomes might include 
satisfactory school performance, and improved school attendance, as well as a lower 
incidence of special education, remedial placement, and retention in grade. 
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Section Three 

Summary of Study Design 



This section lists the research questions addressed by the National Even Start evaluation 
and summarizes the overall approach to the evaluation. Additional information on the 
design is contained in the first and second year reports from this study {St. Pierre et al , 
1991, 1993). 

Research Questions for the National Even Start 
Evaluation 

Presented below is a comprehensive set of research questions that have guided the 
evaluation. The list has evolved over time. Some questions have faded in importance, 
and new ones have been generated as more was learned about how projects are 
implemented. The research questions are organized into four major categories: 

What are the characteristics of Even Start participants? Who is in the program? 

How are Even Start projects implemented and what services do they provide? 
What does the program look like? 

What Even Start services are received by participating families? What do families 
receive by participating? 

What are the effects cf Even Start projects on participants? (What difference does 
Even Start make in the lives of participants? 

These questions build on those listed in the RFP for this study, as well as on the 
conceptual model that was presented in *he previous section of this report. 



What are the Characteristics of Even Start Participants? 

This set of questions calls for a thorough description of the demographic characteristics 
of Even Start participants. 

What are the characteristics of families, adults, and children who are served 
by Even Start (e.g., gender, ethnicity, race, presence of handicapping 
condition, primary languages, educational status, employment status)? 
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What proportion of Even Start participants are from sp' .;ial populations 
(e.g., handicapped, limited-English-proficient)? 

What social services were received by Even Start participants prior to entry 
into Even Start? What non-Even Start services are received by Even Start 
participants? 



How are Even Start Projects Implemented and What Services do They 
Provide? 

This set of questions focuses on the services being provided by Even Start projects and 
the ways in which Even Start projects are being implemented. Questions deal with the 
geographic distribution of projects, the use of available curricula/models, the cost of Even 
Start and the allocation of Even Start funds, recruitment and screening of families, 
characteristics and training of Even Start staff, collaborative efforts of Even Start projects, 
and barriers to program implementation. 

How are Even Start projects distributed by geographic location and 
urbanicity? 

Are Even Start projects designed as year-round or school-year projects? 

To what extent do Even Start projects use available curriculum materials or 
program models? What materials and models are frequently used? 

What is the cost of Even Start projects? How are Even Start funds 
allocated within projects? What proportion of Even Start funds is spent on 
different activities? How are local contribution funds obtained and used? 

Do projects target special groups of families? What procedures are used to 
recruit eligible families? What procedures are used to screen and assess 
parents' and children's needs? 

What is the background and training of Even Start staff? What is the 
proportion of professional, paraprofessional, and volunteer staff? In what 
topic areas does Even Start provide staff training? How much training is 
provided to Even Start staff (and to staff at other agencies) in each area? 

What types of collaboration exist between Even Start and other agencies? 
What types of agencies are collaborating with Even Start projects? What 
mechanisms are being used to enhance the cooperation/collaboration 
(formal letter of agreement, informal agreement, increased communication, 
etc.)? 

What core, support, and special services are provided by Even Start 
projects? What services do collaborating agencies provide? 

What barriers exist to successful program implementation? 
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What Even Start Services are Received by Participating Families? 

This set of questions deals with the Even Start "treatment" that is received by 
participating families. 

How much time do Even Start participants spend in each core service? 
Which core services (and how much of each service) are provided to 
parents and children together? Which core services (and how much of each 
service) are provided in the home? 

How long do families participate in Even Start? 



What are the Effects of Even Start Projects on Participants? 

This set of questions deals with the impact of Even Start projects. Questions concern 
areas such as effects on participating children, effects on parents and families, the 
relationship between amount of services and child/parent/family effects, and the 
effectiveness of different Even Start models. 

At entry to Even Start, how do the school-readiness and literacy-related 
skills of Even Start children compare with the school-readiness and literacy- 
related skills of children in other early childhood education programs for the 
disadvantaged? Of children nationally? 

At entry to Even Start, what is the level of basic skills and/or English- 
speaking ability of participating parents? What is their educational 
attainment? What are parents' educational expectations for their children 
and for themselves? What types of parent-child interactions do parents 
engage in? How involved are parents in their children's education? What 
are parents' ideas about child-rearing practices? 

What are the effects of Even Start on the school-readiness and literacy- 
related skills of Even Start children? After participating in Even Start, how 
do the school-readiness and literacy-related skills of Even Start preschool 
children compare with the school-readiness and literacy-related skills of 
disadvantaged children in other early childhood education programs? With 
children nationally? 

What are the effects of Even Start on the basic skills and/or English- 
speaking ability of participating adults? On parent-child interactions, parent 
behaviors, parent expectations, and parenting skills? On parent educational 
status, receipt of a GED, participation in job training or further education, 
and/or job placement? 
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What is the relationship between amount of home-based services, amount 
of parent/child together services, length of participation, and outcomes for 
children? Outcomes for parents? 

How do parents' attitudes/expectations, basic skills, and patterns of parent- 
child interactions relate to children's school readiness or achievement? 

Do adults participating in Even Start have better retention and/or attendance 
in ESL or ABE programs than adults in regular adult education programs? 

Based on information about the services provided, is it possible to identify 
a set of Even Start "models" that exhibit variation in design and service 
delivery? Are some Even Start models more effective than others in terms 
of enhancing adult basic skills, children's school readiness, and parents' 
behaviors and expectations? Are some Even Start models particularly 
cost-effective? 

Across Even Start projects, are there practices or components that are 
particularly effective? 



Components of the National Even Start Evaluation 

A four-component evaluation has been designed in order to address the questions listed 
above {see Exhibit 3.1). The components are: (1) the National Evaluation Information 
System (NEIS) for all Even Start projects, (2) an In-Depth Study of ten projects, (3) other 
local evaluations conducted by individual projects, and {4} local application for approval 
by the Department of Education's Program Effectiveness Panel (PEP) to enter the National 
Diffusion Network (NDN). 



National Evaluation Information System 

The first component of the evaluation is the National Evaluation Information System 
(NEIS), which is designed to collect a common set of data from each Even Start project 
and from most Even Start participants. The purpose of the NEIS is to provide ongoing 
descriptive information about the Even Start program, including the types of projects that 
have been funded, the services provided, the collaborative efforts undertaken, and the 
obstacles that exist to program implementation. The NEIS is structured to provide 
detailed information describing the families that participate in Even Start, the services they 
receive, and the progress they make in areas such as adult basic skills, children's school 
readiness and literacy-related behaviors, and parent-child interactions. 
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In-Deptk Study of Ten Projects 



The second component of the evaluation is the In-Depth Study. This component is 
designed to complement the broad-based data collected from all Even Start projects 
through the NEIS by providing more in-depth information on a subset of ten purposively- 
selected grantees. Whereas the NEIS provides common data on all projects, the In-Depth 
Study evaluates a subset of projects in more detail, including random assignment of 
participants to treatment and control groups. This methodology will provide especially 
credible evidence regarding the effectiveness of the Even Start approach. The In-Depth 
Study focuses on short-term outcomes of Even Start for parents and children and on the 
relationship between services received and outcomes. 



Other Local Evaluation Activities 

After they have met requirements for the National Evaluation Information System and the 
In-Depth Study, grantees may conduct other local evaluation activities that they think are 
necessary or appropriate. Local evaluation activities can be funded through the projects' 
evaluation budget, but must be approved by the State Department of Education, typically 
through the continuation grant. 



Local Application for PEP/NDN Qualification 

The final component of the evaluation is primarily the responsibility of individual Even 
Start grantees. In accordance with Section 1058(c) of the Even Start legislation, Even 
Start projects should submit evidence of their effectiveness for approval by the 
Department of Education's Program Effectiveness Panel (PEP). Approval by PEP results 
in entry to the National Diffusion Network (NDN) and national recognition as a model 
program worthy of emulation. After entry to NDN, the project may apply to NDN for 
additional dissemination funds as a developer/demonstrator project. 

Starting in 1 993, Even Start projects can also apply to a new NDN dissemination center 
on family literacy and obtain approval and recognition of their program. The National 
Center for Family Literacy in Louisville, Kentucky, now has authority to identify 
exemplary family literacy projects for inclusion as National Diffusion Network projects. 
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Section Four 

Characteristics of Even Start Participants 



This section presents data from NEIS that describe the families, children, and adults that 
participated in Even Start during the 1991-92 program year in terms of household 
composition, race and ethnicity, educational and employment status, primary language, 
and other variables. Two sets of projects participated in the evaluation: 76 projects 
which were first funded in 1989 (Cohort 1 projects) and 47 projects which were first 
funded in 1 990 (Cohort 2 projects). Most tables and analyses in this report combine data 
across the two cohorts. 



Number of Participants 

Even Start provides three types of educational services (i.e., adult education, parenting 
education and early childhood education) and several types of support services (e.g., 
transportation, counseling). Exhibit 4.1 shows that during 1991-92, Cohort 1 and Cohort 
2 projects provided some core services to 9,690 families, an average of 81 families per 
project. Projects provided early childhood education services to 13,541 children (1 14 per 
project), and adult education or parenting education services to 10,800 parents (91 per 
project) in these same families. 



Characteristics of Participating Families 

Family characteristics discussed here are based on data from the 9,690 families in which 
at least one family member participated in some Even Start core service during the 199 V 
92 program year. The data represent 119 out of 123 Cohort 1 and Cohort 2 projects 
(four projects did not provide the necessary data). 



Household Composition 

The largest percentage of families participating in Even Start described themselves as 
couples (48 percent), followed by single parent households (39 percent). The remaining 
categories included extended families (12 percent) and "other" families (1 percent). The 
latter category encompasses children living with grandparents, step-parents or guardians, 
or nonrelated children for whom the Even Start adult was the primary caregiver (Exhibit 
4.2). 
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Exhibit 4.1 


j 


Number of Families. Children and Parents 
Participating in Even Start 
(1991-92 Program Year) 




Total 

Type of Participant || n 


Mean N per 
Project 


Families receiving some core service 9,690 
Children receiving ECE 13,541 
Parents receiving ABE or PE 1 0,800 


81 
114 
91 


Note: Based on reports from 119 Cohort 1 and Cohort 2 projects. 




Exhibit reads: 9,690 families participated in some Even Start core service in 1991-92- 



Exhibit 4.2: Structure of Even Start Families 
(1991-92 Program Year) 



[ Couple 

j 




— Other 
Extended family 



Exhibit reads: 48 percent of Even Start families were couples with children. 
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Consistent with descriptions of family structure, the majority of Even Start households 
included two adults (52 percent), followed by households with one adult (36 percent). 
Seven percent of households included three adults, and 6 percent included four or more 
adults (Exhibit 4.3). Most households included one (21 percent), two (34 percent) or 
three (25 percent) children. Twelve percent of Even Start families included four children, 
and 8 percent included five or more children (Exhibit 4.4), 

The top portion of Exhibit 4.5 shows that 18 percent of Even Start families had at least 
one child less than one year old. The middle portion of the exhibit shows that the 
majority of families had either one (43 percent) or two (38 percent) children between the 
ages of one and seven. Fourteen percent of households included three eligible children, 
and 5 percent of households included four or more children. The bottom portion shows 
that 37 percent of households included some children too old to participate in Even Start 
(ages eight through 16), 



Household Income 

An examination of the sources of financial support for Even Start families during the 
1991-92 program year shows that the primary sources of financial support were 
government assistance (52 percent) and wages from jobs (47 percent). Only 3 percent 
of families used alimony or child support as a primary source of support (Exhibit 4.6). 



■ 

Exhibit 4.3: Number of Adults in Even Start Families 
(1991-92 Program Year) 




Two adults 



Exhibit reads: 35.6 percent of Even Start families had one adult in the household. 
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Exhibit 4.4: Number of Children in Even Start Families 
(1991-92 Program Year) 



One child 




Exhibit reads: 20.7 percent of Even Start families had one child in the household. 



Exhibit 4.5 

Percentage of Even Start Families by 
Number of Children in Different Age Categories 
{1991-92 Program Year) 


Age Category 


II % 


Children less than 1 year (eligible) 




None 


82% 


One 


18% 


Total Families 


100% 


Children ages 1-7 years (eligible) 




One 


43% 


Two 


38% 


Three 


14% 


Four or more 


5% 


Total families 


100% 


Children ages 8-16 years (ineligible) 




None 


63% 


One 


22% 


Two 


11% 


Three 


3% 


Four or more 


1% 


Total families 


100% 


Exhibit reads: 82 percent of Even Start families had no children less than one year of age. 
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Exhibit 4.6 




Primary Sources of Financial Support for Even Start Families 
(1991-92 Program Year) 


Primary Sources of Financial Support 


% 


Government assistance 


52% 


Job wages 


47% 


Alimony/child support 


3% 


Other 


4% 


Note: Percentages sum to more than 100 percent because multiple responses were allowed. 


Exhibit reads: 52 percent of Even Start families relied on government assistance as their primary source 
of financial support in 1991-92. 



The median total annual household income reported by Even Start families was well under 
$10,000: 40 percent of families reported incomes under $5,000, 29 percent had income 
between $5,000 and $10,000, 16 percent reported incomes in the $10,000 to $1 5,000 
range, 8 percent had incomes between $15,000 and $20,000, and only 7 percent 
reported incomes over $20,000 (Exhibit 4.7}. By way of comparison, 33 percent of 
Chapter 1 families had annua! incomes under $10,000 (U.S. Department of Education 
1993). 

Characteristics of Participating Adults in Even Start 
Families 

Each Even Start family contains one or more adults. Data were reported on a total of 
10,800 adults in the 9,690 Even Start families. 



Age of Participating Adults in Even Start Families 

Most adults in Even Start families were between 22 and 29 years old (47 percent), or 
between 30 and 39 years old (31 percent). Only 12 percent were in the 18 to 21 age 
range, 2 percent were younger than 18, 7 percent of Even Start adults were 40 to 49 
years old, and 2 percent were 50 or older (Exhibit 4.8). Given the low-income population 
targeted by Even Start, it might be expected that more than two percent of Even Start 
adults would be under 1 8 years of age. However, a family is eligible for Even Start only 
if an adult in the family qualifies for adult basic education, and adult basic education 
participants must either be at least 16 years old and not in school or beyond the age of 
compulsory schooling in their state. 



Even Start 



Characteristics of Evan 
4-5 2 O Start Participants 



Exhibit 4.7: Total Annual Income of 
Even Start Families (1991-92 Program Year) 



-$0-$4,999 



$5.000-$9,99& 




$25,000 or more 

$20,000-$24,999 

315,000-$1 9,000 



\ 



$10;00041*§§3 



Exhibit reads: 39.7 percent of Even Start families had annual incomes of less than $5,000 in 1991-92. 



Exhibit 4.8: Age of Participating Adults in Even Start Families 
(1991-92 Program Year) 



40*49 years old 




Exhibit reads: 1 1 .6 percent of participating adults in Even Start families were 8-21 years old in 1991-92. 

Characteristics of Evan 
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Gender of Participating Adults in Even Start Families 



Exhibit 4.9 displays the gender of adults who participate in Even Start core services. Of 
all adults, 20 percent were male and 80 percent were female. 



Race/Ethnicity of Participating Adults in Even Start Families 

Forty-five percent of Even Start adults were categorized as white and 27 percent were 
African-American (Exhibit 4.10). Six percent of adults categorized themselves as Native 
American, and 3 percent as Asian/Pacific Islander or other. Eighteen percent did not 
specify a racial category but listed Hispanic as their ethnic heritage. Exhibit 4.1 1 shows 
that, of this group, 80 percent listed their background as Mexican, 6 percent listed Puerto 
Rican, and 14 percent selected "other Hispanic. " 



Educational Attainment of Adults in Even Start Families 

A distribution of years of educational attainment prior to participating in Even Start is 
shown in Exhibit 4.12. The solid line represents adults participating in any type of Even 
Start core service. The dashed line represents nonparticipating adults from families in 
which at least one adult participated in some type of Even Start core service. 

In general, nonparticipants in Even Start families had a higher level of education than 
participating adults. This makes sense since each participating adult ought to be in need 
of adult education, whereas nonparticipating adults may or may not have such a need. 
A total of 77 percent of core service participants and 64 percent of nonparticipants did 
not graduate from high school. An additional 17 percent of participants and 27 percent 
of nonparticipants either had a high school diploma or a GED, and 6 percent of 
participants and 9 percent of nonparticipants had undertaken some postsecondary 
education. 

Program regulations allow adults to participate in adult basic education even if they have 
a high school diploma or a GED. Participation is based on educational need rather than 
diploma status. Also it should be noted that ESL adults may have completed secondary 
or postsecondary programs in their native countries but still require adult education to 
improve their English skills. 
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Exhibit 4.9: Gender of Participating Adults in Even Start Families 
(1991-92 Program Year) 




Exhibit 4.10: Racial/Ethnic Background of Participating Adults 
in Even Start Families (1991-92 Program Year) 
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Exhibit 4.1 1 : Ethnic Background of Hispanic Adults in Even Start Families 

(1991-92 Program Year) 




Exhibit 4.12: Years of Schooling for Adults in Even Start Families: 
Core Service Participants and Non-Participants (1991-92 Program Year) 

Legend j 
Core Participants ! 




v ■ : : : ! 1 : 1 1 1 1 1 1 

None 1 2 3 4 5 6 7 8 9 10 11 12 HS GEO PS Otr 

Years of Schooling 



Exhibit reads: Nonparticipants in Even Start core services tended to have higher levels of education than 
participating adults in the same family. 
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Functional Literacy Levels of Even Start Adults 



This evaluation measured the functional literacy of Even Start adults using the 
Comprehensive Adult Student Assessment System (CASAS), an adult-oriented measure 
of a broad range of adult literacy skills and their application in real life domains including 
consumer economics, government and law, occupational knowledge, community 
resources, and health. Details on the CASAS, including a description of the different 
levels of achievement, are contained in Section Nine of this report. 

The average Even Start adult enters with high school literacy skills. The mean CASAS 
pretest score was 229 scale score points with a standard deviation of 14 points {Exhibit 
4.13). Nearly three-quarters of the participants (71 percent) attained pretest scores of 
225 or more~a literacy level equivalent at least to entry to high school. One in six (17 
percent) entered with an intermediate level of literacy. Very few adults entered with a 
basic literacy level (8 percent) or a beginning level (5 percent). However, because adults 
were not tested unless their primary language was English and a few sites did not test 
adults with reported learning disabilities, it is possible that these figures overestimate the 
average literacy levels of Even Start adults. 
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Exhibit 4.1 3: Distribution of CASAS Functional Literacy Level 
for Entering Adults (NEIS Data Set) 
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High School 



Exhibit reads: 71 percent of Even Start adults had CASAS pretest scores that placed them in the "high school" 
functional level. 
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These pretest scores are consistent with the pre-Even Start level of education reported 
for Even Start adults: 23 percent of the adults participating in Even Start core services 
never attended high school, 54 percent had some high school education, and an additional 
23 percent had a high school diploma or a GED, Thus, according to self-reports, 77 
percent of Even Start adults entered with at least some high school experience; and 
according to the CASAS pretest, 71 percent of Even Start adults entered with at least 
high school entry reading skills. 

Exhibit 4.14 shows average CASAS pretest scores for selected subgroups of adults who 
took the CASAS pretest within one month of entering Even Start. Adults entering Even 
Start with less than a fifth grade education averaged 207 scale score points on the 
pretest, equivalent to a basic level of functional literacy. Those entering with a ninth to 
twelfth grade education scored 230 points, a high school functional level. Participants 
entering with a GED or diploma scored no better at 231, suggesting that these adults 
could benefit from Even Start services despite holding a diploma. 

The pretest literacy level of adults was generally a little lower for older adults than for 
those under age 25. Not surprisingly, adults for whom English was the primary language 
scored much higher on the pretest (231) than those for whom English was not the 
primary language (214). Adults in families with higher incomes also tended to enter with 
higher literacy levels than adults in families with lower incomes. 

Exhibit 4.15 compares pretest scores on the CASAS reading achievement test of Even 
Start adults with the pretest scores of adults in other related programs. In one evaluation 
of the "321 " Program (a special adult basic education program in California), researchers 
(CASAS, 1 992) reported pretest means of 21 7 scale score points for 5,029 ABE students 
and 210 points for 31,006 ESL students, lower entry levels than their Even Start 
counterparts. The California programs served a more ethnically diverse population than 
Even Start, often with more limited English proficiency, and many participants enrolled to 
meet a state or federel mandate. One explanation for the lower scores among California 
321 participants is that they required tests for adults whose first language was not 
English, whereas the Even Start evaluation did not include such a requirement, eliminating 
these potentially low scorers from the study. 

The California-funded GAIN (Greater Avenues for Independence) program for Aid to 
Families with Dependent Children applicants and recipients conducted literacy testing on 
all participants. An early study on this program reported an average pretest of 233 
(CASAS, 1990). The higher literacy level of GAIN adults was not surprising since more 
than half of the participants of this welfare reform program had high school diplomas, a 
GED, or other degrees before entering the program. 

Although the average entry literacy level of Even Start participants is comparable to other 
adult education programs, as reported above, there is considerable variation among 
individual projects. Exhibit 4.16 shows the distribution of average CASAS pretest scores 
across the 97 projects that reported valid scores on at least five adults. While two-thirds 
(67 percent) of the projects had average pretest scores within the high school functional 
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Exhibit 4.14 



Pretest Scores on the CASAS Reading Survey 

(Scale Score Points; NEIS Data Set) | 


Group 


Number of 
Adults 


Average 
Pretest 
Mean 


Pretest 
Standard 
Deviation 


Highest grade at intake 








Grades 0-4 


43 


207.4 


23.8 


Grades 5-8 


398 


224.0 


16.7 


Grades 9-1 2 


1,727 


230.3 


12.7 


Diploma/GED 


410 


230.9 


14.0 


Age at intake 








16-20 


377 


230.1 


12.8 


21-25 


881 


231.0 


12.6 


26-30 


698 


228.4 


15.6 


31-35 


312 


228.7 


14.0 


35-40 


136 


225.6 


17.6 


Over 40 


115 


222.1 


18.7 


Primary language is English 








Yes 


2,180 


231.3 


12.2 


No 


321 


214.3 


18.5 


Family annual income 








Under $5,000 


1,134 


227.8 


14.3 


$ 5,000 - 10,000 


668 


229.5 


13.5 


$10,000 - 15,000 


289 


229.0 


16.0 


$15,000 - 20,000 


176 


231.4 


14.7 


$20,000 - 25,000 


95 


233.4 


15.1 


Over $25,000 


83 


235.9 


15.8 


TOTAL 


2,587 


229.0 


14.4 


| Exhibit reads: Adults entering Even Start without a high school diploma or GED performed at their | 
| expected functional level on their CASAS pretest. However, the 410 adults entering with a diploma or 
| LZD performed no better than those adults who had reached grades 9-12. This confirms that their | 
| placement in Even Start was appropriate. j 
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Exhibit 4.15: Average Pretest Scale Score on the CASAS Reading Survey 
for Even Start and Three Adult Literacy Programs 




Ev«n Start CAGain CA 321 ABE 



Exhibit reads: The average entry literacy level for Even Start adults, as measured by the CASAS Reading 
Survey, was slightly lower than participants in the Greater Avenues for Independence (GAIN) programs in 
California but higher than the ethnically diverse participants of the federal 321 -funded adult education programs. 



Exhibit 4.1 S: Project Variation in 
CASAS Entry Scores (NEIS Data Set) 




<2O0 200-214 215-224 225+ 



Mean CASAS Pretest 



Exhibit reads: The CASAS pretest average for three projects fell below 200, the beginning literacy level. The 
average for most projects fell above 225 , the high school literacy level. 
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level, the project averages ranged from 184 at the beginning literacy level to 250 which 
is near the highest valid score in the high school literacy level. Clearly there are 
implications for selecting curriculums and setting goals for outcomes between those 
projects with most adults entering at an elementary or middle school academic skills level, 
compared to those with more adults at low high school levels. 



Primary Language of Participating Adults 

English was reported as the primary language for 72 percent of participating adults, 
Spanish was the primary language for 22 percent, and 6 percent reported some other 
primary language including Hmong, Vietnamese, Chinese, Creole, French, and others 
(Exhibit 4.17). 

Exhibit 4.18 displays the reported ability of adults for whom English is not the primary 
language to speak, read and understand English. About 18 percent reported the ability 
to speak English "very well," 49 percent could speak English "somewhat," and 33 percent 
"not at all." Seventeen percent could read English "very well," 45 percent "somewhat," 
and 38 percent "not at all." Finally, three-quarters of adults participating in Even Start 
for whom English is not the primary language understand English "very well" (22 percent) 
or "somewhat" {53 percent). The remaining 25 percent understood English "not at all." 

Exhibit 4.19 presents information about the language used by adults when reading to their 
children. It can be seen that 70 percent of Even Start adults for whom English was not 
the primary language reported that they read to their children in their primary language, 
20 percent of this group read to their children in English, and 8 percent read in both 
English and their primary language. 



Employment Status of Participating Adults in Even Start Families 

Most Even Start adults were unemployed at the start of the program year. Seventy-two 
percent of the adults who participated in Even Start core services were unemployed, 20 
percent were employed full-time, and 8 percent were employed part-time (Exhibit 4.20). 
Exhibit 4.21 expands on this information by showing the duration of employment or 
unemployment. Seventy-four percent of the adults who were unemployed had been so 
for more than 12 months. Similarly, over half (57 percent) of the adults who were 
employed had been so for more than 12 months. 
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Exhibit 4.1 7: Primary Language of Adults in Even 
Start Families (1991-92 Program Year) 




Other 



Exhibit reads: 72 percent of Even Start adults reported English as their, primary language. 





Exhibit 4.18 




Reported English Language Facility of Adults Participating in 
Even Start Core Services, 
For Adults Whose First Language is Not English 
(1991-92 Program Year) 




Reported English Language Facility || 


% 


Speaks English 






Very well 




18% 


Somewhat 




49% 


Not at all 




33% 


Reads English 






Very well 




17% 


Somewhat 




45% 


Not at all 




38% 


Understands English 






Very well 




22% 


Somewhat 




53% 


Not at all 




25% 



Exhibit reads: Of adults whose first language is not English, 18 percent report speaking English very 
well. 
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Exhibit 4.19 




Language Used to Read to Child, 
For Adults Whose First Language is Not English 
(1991-92 Proaram Year) 




Language usea to Head to uniid 


% 


Primary language (non-English) 


70% 


English 


20% 


Both 


8% 


Other 


2% 


Total adults 


100% 


Exhibit reads: Of adults whose first language is not English, 70 percent read to their child in their 
primary language. 



Exhibit 4.20: Employment Status of Adults in Evan 
Start Families (1991-92 Program Year) 



Employed part-time 



Unemployed 




Employed full-time 



Exhibit reads: 72 percent of Even Start adults were unemployed in 1991-92. 
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Exhibit 4.21 

Duration of Employment Status of Adults in Even Start Families 
(1991-92 Program Year) 


Employment Status/Duration 


II 


N 


I % 


Employed 








Less than 6 months 




636 


25% 


6 to 1 2 months 




456 


18% 


More than 12 months 




1,421 


57% 


Unemployed 








Less than 6 months 




997 


15% 


6 to 1 2 months 




670 


11% 


More than 1 2 months 




4,590 


74% 


Exhibit reads: Of the unemployed Even Start adults, 74 percent had been unemployed for more than 12 
months. 



Characteristics of Pai"ticipating Children 

This section of the report presents data on 13,541 children in families that participated 
in core Even Start services during the 1991-92 program year. There are no meaningful 
differences between the child-level variables discussed here and the adult-level variables 
just discussed. 



Age of Participating Children 

Even Start projects focus somewhat on children in the middle of the eligible age range 
(birth through age 7): 53 percent are three, four, or five years of age; 31 percent are 
less than three, and 16 percent are six or seven years old {Exhibit 4.22). 



Gender of Participating Children 

The percentage of male and female participating children is shown in Exhibit 4.23: 51 
percent of the children are male and 49 percent are female. 
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Exhibit 4.22: Age of Participating Children in Even Start Families 
(1991-92 Program Year) 



6 to 7 years old 




7 to 8 years oW 

0 to 1 year oW 



5 to 6 years old- 



7 \ i / / 

14.9% \ : / / 10 - 8% 



-1 to 2 years old 



4 to 5 years okt 




2 to 3 years old 



3 to 4 years old 



Exhibit reads: Six percent of participating Even Start children were less than one year old in 1991-92. 



Exhibit 4.23: Gender of Participating Children in Even Start Families 
(1991-92 Program Year) 




•MaJe 



Exhibit reads: 49 percent of participating Even Start children were females. 
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Race/Ethnicity of Children in Even Start Families 



Racial and ethnic categories for children are presented in fcxhibit 4.24 and, as expected, 
are similar to those presented for adults: 42 percent of chile ren were identified as white, 
30 percent of children are African-American, 6 percent are Native American, 3 percent 
are Asian/Pacific Islander, and 3 percent are other. For 19 percent of children, Hispanic 
9 percent as Puerto Rican, and 12 percent as "other Hispanic" (see Exhibit 4.25). 



Educational Experiences of Participating Children 

Sixty-four percent of Even Start children were reported to have had no formal educational 
experience prior to the beginning of Even Start, 27 percent had a preschool experience 
{either Head Start or some other preschool), 12 percent had participated in kindergarten, 
and 5 percent participated in a primary grade (Exhibit 4.26). 



Special Needs of Participating Children 

Of all children participating in Even Start core services, nine percent were reported to 
have a disability. Exhibit 4.27 shows the specific types of special needs, none of which 
exist for more than three percent of the Even Start population. Learning problems were 
cited for 37 percent of the special needs children (3 percent of all Even Start children), 
emotional problems for 1 9 percent, hearing problems for 1 2 percent, and "other" physical 
disabilities for 1 2 percent. 



Vocabulary Skills of Even Start Children 

This evaluation used the Peabody Picture Vocabulary Test (PPVT) to assess children's 
receptive (hearing) vocabulary. PPVT scores are standardized, so that a score of 100 
represents the average score for the norms group at each age level. 

Across children of all ages, pretest scores averaged 81.8 standard score points (Exhibit 
4.28). This corresponds to the twelfth percentile when compared to national norms and 
points out the low verbal skills of children prior to entry into Even Start. Directions for 
test administration were that program staff were to determine whether to administer the 
PPVT (for English speakers) or the TVIP (for Spanish speakers). Children who took the 
PPVT averaged 80.8 on the pretest (tenth percentile) compared with children who took 
the TVIP who averaged 91 .1 on the pretest (27th percentile). It is not possible to make 
comparisons between these percentiles because the norms groups are different for the 
PPVT and TVIP. 
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Exhibit 4.24: F *dal/Ethnic Background of Participating Children 
in Even Start Families (1991-92 Program Year) 



Asian, Pacific Islander. Other- 
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Exhibit reads: 30.3 percent of participating children in Even Start families were reported as being African- 
American. 



Exhibit 4.25: Ethnic Background of Hispanic Children in Even Start Families 

(1 991 -92 Program Year) 



Puerto Ri< 




Exhibit reads: 78.8 percent of Hispanic children in Even Start were identified as being of Mexican background. 
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Exhibit 4.26: Previous Educational Experience of Children 
Participating in Even Start Core Services (1991-92 Program Year) 
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Exhibit reads: 14 percent of Even Start children were in Head Start prior to joining Even Start, 



Exhibit 4.27 




Types of Special Needs for Children Participating in Even Start Core Services 

(1991-92 Program Year) 


Type of Special Need 


% of Children 
with Special Need 
<N= 1,205) 


% of Children in 
Even Start 
(N= 13,541) 


Specific learning problem 


37% 


3.32% 


Emotional problem 


19% 


1 .66% 


Hearing problem 


12% 


1.07% 


Other physical disability 


12% 


1 .05% 


Visual problem 


9% 


0.78% 


Mental retardation 


6% 


0.55% 


Speech problem 


3% 


0.35% 


Orthopedic problem 


0% 


0.01 % 


Other 


24% 


2.15% 


NOTE: Percentages sum to more than 100 percent because multiple responses were allowed. 


Exhibit reads: Nine percent of Even Start children who had a special need have a visual problem. This 
represents 0.78 percent of all Even Start children. 
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PPVT Pretest Scores 
(Standard Scores from the NEIS Data Set) 
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Mean || 


SD 


Age at pretest 






3-0 to 3-1 1 552 


82.3 


15.7 


4-0 to 4-1 1 773 


78.2 


17.7 


5-0 to 5-1 1 337 


84.3 


16.1 


6-0 to 6-11 201 


87.7 


17.4 


7-0 to 7-11 83 


83.4 


16.4 


Gender 






Male 965 


81.5 


17.2 


Female 971 


82.1 


16.9 


Prior preschool experience 






No 1,229 


80.9 


17.2 


Yes 661 


83.0 


17.1 


Highest grade attained by target parent 






Grade 0-4 44 


81.5 


18.2 


Grade 5-8 328 


82.7 


17.5 


Grade 9-12 1,133 


81.0 


17.1 


Diploma or GED 425 


83.7 


16.8 


Language test administered in 






English <PPVT) 1,715 


80.8 


16.9 


Spanish (TVIP) 223 


91.1 


16.4 


Total 2,025 


81.8 


17.1 


Exhibit re^ds: The average PPVT pretest score for males was 81 .5 standard 
points. 


score 
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Comparison With Head Start and CCDP 



Even Start is a comparatively new federal program and it is of interest to see how the 
characteristics of Even Start participants compare to the characteristics of participants 
in other similar federal programs. Such cross-program comparisons are never easy 
because measures are often not comparable across studies. Nevertheless, Exhibit 4.29 
presents selected data on Even Start families, on families that participated in Head Start, 
and on families who are participating in the Comprehensive Child Development Program 
(a family support program funded by the U.S. Department of Health and Human Services). 

Even Start families are iess often headed by a single parent than families that participate 
either in Head Start or in CCDP {36 percent single parent families in Even Start vs. 55 
percent in Head Start and 63 percent in CCDP). Even Start and Head Start appear to 
serve about the same percentage of very low-income families-40 percent of Even Start 
families have incomes below $5,000 while 46 percent of Head Start families have 
incorr.as below $6,000. CCDP seems to serve a somewhat larger percentage of very low 
income families-62 percent of CCDP families have incomes below $6,000. Because 
there is no income cut off, Even Start includes a larger proportion of families with 
relatively higher incomes (over $15,000) than CCDP. . 

Finally, Even Start serves a higher proportion of white families than Head Start or CCDP 
(45 percent, 27 percent, and 24 percent respectively) and a lower proportion of African- 
American families (27 percent, 33 percent, and 44 percent respectively). This latter 
finding could result from the fact that Even Start projects are required to be geographically 
distributed and to include both urban and rural areas, while Head Start grantees 
overrepresent the South, and CCDP overrepresents inner cities. 
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Exhibit 4.29 




Family Characteristics: Even Start, Head Start, CCDP 


Familv ChamptpriQtir 

1 Olillly VI lul avICl I9U v 


even oiarx 


rieaa otart 


CCDP 


Single parent 
households 


36% 


55% 


63% 


Race/ethnicity 








African-American 


27% 


38% 


44% 


White 


45% 


33% 


24% 


Hispanic 


18% 


22% 


27% 


Other 


9% 


7% 


5% 


Mean family income 


$8,1 75 4 


$7,020 3 


$5,707 


Income distribution 


$0-$4,999 = 40% 

$5,000- 
$9,999 = 29% 

$10,000- 
$14,999 = 16% 

over 
$15,000 = 15% 


$0- 

$5,999 = 46% 


$0-$5,999 = 62% 

$6,000- 
$8,999 = 20% 

$9,000- 
$11,999 = 12% 

over $12,000 = 6% 


Administration on Children, Youth and Families (1992). Statistics are for 1991. 
2 Hubbell, R. et al. (1 991 ). Statistics are for 1 989. 
3 Glantz, F. et al. (1988). Statistics are for 1986. 
4 Estimated from income distribution. 
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Section Five 

Characteristics of Even Start Projects 



This section presents date from the NEIS which describe the Cohort 1 and Cohort 2 Even 
Start projects as they were implemented during the 1 991 -92 program year. This was the 
third year of program operations for Cohort 1 and the second year of operations for 
Cohort 2. Data for this section are based on self-reports from project directors . 

The discussion covers the following topics: recruitment and screening, types of core 
services delivered, core services delivered to adults and children together, types of 
support services, types of special activities, cooperative arrangements, implementation 
problems, and technical assistance needs. 



Recruitment and Screening Strategies 



Recruitment Strategies 

Even Start project directors were asked to identify the strategies that worked best for 
recruiting eligible Even Start participants during the 1991-92 program year. Each project 
selected up to three successful strategies from a list of options and wrote in other 
successful strategies. 

The two most successful strategies used by Even Start projects were referrals from local 
agencies and from the public schools, each listed by 59 percent of the projects as a 
successful recruiting strategy (Exhibit 5.1). Over half (51 percent) of the projects listed 
home visits as a successful strategy. Three other approaches to recruiting were listed by 
more than one-third of the projects: telephone contact (41 percent), word of mouth (38 
percent), and referrals from Head Start (37 percent). Targeted mailings were seen as 
successful by 21 percent and the mass media were successful for 1 8 percent of all Even 
Start projects. 



Steps Used in Formal Screening 

Projects were asked to identify the formal steps they used to screen participants by 
responding to a checklist and by writing in additional steps or activities. A summary of 
responses is presented in Exhibit 5.2. Nearly all of the projects verified the eligibility of 
potential participants (97 percent) and gave a basic orientation (89 percent) during the 
screening process. More than half of the projects assessed the basic skills of adults (65 
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Exhibit 5.1 




Successful Strategies for Recruiting Eligible Participants: 
(1991-92 Program Year) 


Recruiting Strategy 


Percent of Projects 


Aqencv referrals (e.a., WIC health clinic) 


Ov7 /0 


Public school referrals 


/0 


Home visits to ootential narticinants 


*J I /o 


Teleohone contact 


*f 1 /o 


Word of mouth 


OO /o 


Referrals from Head Start staff 


o / /o 


Targeted mailings 


21 % 


Mass media 


18% 


Joint efforts with collaborative agency 


9% 


Posters / flyers 


6% 


I Presentations / visits to community agencies 


6% 


General result of greater community visibility 


5% 


Even Start recruiting program 
II (open house, fun fair, etc.) 


2% 


I Exhibit reads: 51 percent of the reporting projects identified home visits as a successful recruiting 
|| strategy. 



Exhibit 5.2 




Steps Included in Formal Screening of Potential Participants 
(1991 -S2 Program Year) 


Screening Activity 


Percent of Projects 


Verify eligibility 


97% 


Orientation 


89% 


Assess basic skills of adults 


65% 


Contact other agencies 


58% 


Test children 


35% 


Counseling 


24% 


None 


1% 


Exhibit reads: 97 percent of reporting projects verified eligibility as a step in format screening of 
potential participants. 
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percent) and contacted other agencies as part of the formal screening (58 percent). One- 
third of the projects (35 percent) tested children as part of screening, and 24 percent 
provided some counseling during screening. 



Core Services Delivered: Types and Providers 

Core services may be provided by staff funded by Even Start or by staff funded by 
cooperating agencies (e.g., a local Head Start program). Consequently, Even Start project 
directors were asked to report the types of core services provided to Even Start 
participants by staff funded through Even Start, by staff supported by cooperating 
agencies, or by both Even Start staff and cooperating agency staff. 



Parenting Education Services 

Even Start projects delivered a wide range of services to help parents raise their children. 
Almost all projects helped families make use of services provided by other social agencies, 
emphasized parents' role in the education of their children, oriented parents and children 
to school routines, furnished information about child development, trained parents in child 
behavior management, worked on building parental self-esteem, and instructed parents 
in life skills and in principles of health and nutrition. 

More than 90 percent of the Even Start projects offered all of the parenting education 
services identified in Exhibit 5.3. Depending on the specific type of parenting education 
service, 34 to 55 percent of the projects provided services exclusively through Even Start 
staff; 36 to 54 percent of the projects delivered services jointly by Even Start staff and 
by staff from cooperating agencies; and about 5 to 10 percent of all projects provided 
services completely through cooperating agencies. In a small percentage of projects 
(fewer than 10 percent) certain types of parenting education services were not provided 
at ail. 



Adult Education Services 

Exhibit 5.4 summarizes the types of adult education services seen in Even Start projects. 
Almost all projects (98 percent) reported that they offered services to prepare adults to 
attain a GED, 86 percent of all projects provided services in adult education, and 92 
percent provided services in adult secondary education. Instruction in English as a second 
language was available ir 39 percent of the projects. 

The locus of responsibility for adult education services differs from that of parenting 
education. Except for ESL services, about one-third of the projects provided some of the 
adult education services directly by Even Start staff, another one-third of the projects 
provided services by cooperating agencies, and about one-third of the projects offered 
services jointly through Even Start and cooperating agency staff. 
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Exhibit 5.3: Types of Parenting Education Services Reported 
(1 991 -92 Program Year) 
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Exhibit reads: Almost half of the Even Start projects used their own resources to provide parenting education 
in life skills. 
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Exhibit 5.4: Types of Adult Education Services Reported 
(1991-92 Program Year) 
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Exhibit reads: Almost all projects provided GED preparation services; about one-third directly through Even 
Start, about one-third through a cooperating agency, and about one-third jointly by Even Start and a cooperating 
agency. 
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Early Childhood Education Services 



Children in Even Start projects received a range of early childhood education services, as 
can be seen in Exhibit 5.5. Three different preschool options were used, with many 
projects using combinations: (1) 72 percent of the projects enrolled some of their children 
in Head Start; (2) 46 percent of the projects enrolled some of their children in a Chapter 
1 pre-K program; and (3) 93 percent of the projects provided some other preschool 
option. For children old enough to be in the public schools, most Even Start projects 
participated in joint planning activities with the public schools. Hence, 85 percent of the 
projects included kindergarten as an Even Start service, and 74 percent of the projects 
provided early childhood education services to children under eight years of age who were 
in primary grades, again through the vehicle of joint planning with the public schools. 

As would be expected, all Head Start and Chapter 1 pre-K services were provided by 
cooperating agencies, as were most kindergarten and primary school services. About 38 
percent of the projects provided "other preschool- services directly by Even Start staff. 
This reliance on existing providers is not surprising given the high cost of early childhood 
education services and their availability through cooperating agencies and the public 
schools. 

Thus, Even Start projects are most likely to participate in the direct provision of services 
for parenting education and are more likely to delegate provision of services for adult 
education and for early childhood education. This fits with Even Start's mandate to build 
on existing services. In most communities, programs for early childhood education and 
of adult basic education already exist, and Even Start projects are taking advantage of 
these ongoing programs. On the other hand, since programs for parenting education are 
much less likely to exist, Even Start projects are focusing their resources in this area. 

Provision of Core Services to Adults and Children 
Together 

Even Start grantees are required to provide some core services to parents and children 
jointly. This is an important part of the Even Start model in that it impresses on parents 
that they are a key to their child's education, and provides opportunities for parents to 
learn and practice skills in working and playing with their children. It allows project staff 
a chance to offer concrete suggestions to parents as well as guidance and support. 
Finally, children are able to see that their own parents are important teachers. 

More than 90 percent of all projects provided each of the following adult/child activities: 
reading and story telling, developing readiness skills, social development and play, 
development of gross motor skills, working with numbers, arts/crafts, and 
health/nutrition {Exhibit 5.6). Writing activities for parents and children together were 
undertaken by 85 percent of the projects, and computer-related activities were provided 
by 64 percent of the projects. 
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Exhibit 5.5: Types of Early Childhood Education Services Reported 
(1991-92 Program Year) 
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Exhibit reads: 72 percent of all Even Start projects enrolled some of their children in Head Start. 



Exhibit 5.6: Types of AdultfChlld Services Reported 
(1991-92 Program Year) 
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Exhibrt reads: Most aduK/chikJ services were provided directly by Even Start rather than by a cooperating 
agency. 
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It is rare that cooperating agencies are solely responsible for delivering core services using 
the adult/child service mode. Instead, it is more likely that activities for adults and 
children together are delivered solely by Even Start or bv Even Start in conjunction with 
a cooperating agency. Depending on the activity, more i. an half of the projects provided 
adult/child services directly, about 40 percent shared responsibility with a cooperating 
agency, and only about 5 percent delegated exclusive provision of adult/child activities 
to an external agency. 



Pattern of Core Services 

Core services are offered using two primary patterns: (1) year-round services and (2) 
services delivered only during the school year. Some projects offered variations on these 
patterns, such as special month-long courses during the summer. Exhibit 5.7 shows that 
68 percent of all projects provided parenting education year-round while 19 percent 
offered it only for the regular school year; 59 percent of all projects provided adult 
education year-round, while 24 percent offered it for the regular school year; and 54 
percent of all projects provided early childhood education year-round, while 27 percent 
offered it for the regular school year. 



Support Services: Types and Providers 

As defined for this study, support services are activities provided directly to Even Start 
families to enable them to participate in core services. Support services remove barriers 
that, if unattended, restrict a family's ability to receive instructional and educational 
services. Such activities as staff development and training, while enabling the project to 
provide effective services to its clients, are not considered support services because 
families are not the direct recipients. 

To avoid duplication, Even Start projects are expected to obtain support services from 
existing sources as much as possible. As is seen in Exhibit 5.8, more than 85 percent of 
the projects provided a wide range of support services including transportation, health 
care assistance, meals, family advocacy assistance, nutrition services, referrals for 
employment services, counseling services, child care, and mental health services. Many 
other support services were provided by at least half of the projects including referrals for 
child protective services and for battered women, treatment for chemical dependency, 
and referrals for services needed by persons with disabilities. 

Across all types of support services, an average of 22 percent of Even Start projects 
provided the service directly, cooperating agencies provided support services in 27 
percent of the projects, and Even Start and cooperating agencies jointly provided another 
28 percent. However, as might be expected, there is substantial variation in the extent 
to which different support services are provided by Even Start or by cooperating agencies. 
The percentage of Even Start projects providing support services with their own funds 
ranged from less than ten percent for health careand^ervices for persons with disabilities 
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Exhibit 5.7 




Term of Year Core Services Are Offered 
(1991-92 Program Year) 




Core Service / Term of Year 


Percent of 
Projects 


Parenting eduction 




Year-round 


68% 


Regular school year 


19% 


Other 


12% 


Adult education 




Year-round 


59% 


Regular school year 


24% 


Other 


16% 


Early childhood education 




Year-round 


54% 


Regular school year 


27% 


Other 


19% 


Exhibit reads: 68 percent of all reporting projects provide year-round parenting education. 



to more than 40 percent for transportation and child care. These findings suggest that 
Even Start projects did, as planned, obtain many support services from existing providers 
and stepped in to provide services not available locally. 

Special Activities 

In addition to providing core and support services, Even Start projects offer other 
occasional or one-time activities for the families they serve. These special activities are 
used to recruit families, provide information or training, celebrate participant 
accomplishments, and promote family pride, unity and sense of belonging. Exhibit 5.9 
lists several types of special activities. The categories are based on a content analysis 
of the written responses of projects in both cohorts. Because projects' responses were 
prompted by an open-end question, the percentage of projects reporting each type of 
activity is not an accurate indicator of the popularity of each type of activity. Projects 
may not have reported all special activities since they were on their own to decide which 
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Exhibit 5.8 






Percentage of Projects Providing Support Services and 
Other Special Activities Through Even Start and/or Cooperating Agencies 

(1991-92 Program Year) 


support services 


Even Start 


Both 


Cooperating 
Agency 


Neither 


Transportation 


42% 


41% 


14% 


4% 


Health care 


8% 


20% 


65% 


8% 


Meals 


24% 


27% 


38% 


11% 


Family advocacy 


35% 


44% 


11% 


11% 


Nutrition 


15% 


42% 


31% 


13% 


Employment 


34% 


43% 


20% 


13% 


Counseling 


16% 


40% 


30% 


14% 


Child care 


45% 


27% 


15% 


14% 


Mental health 


24% 


48% 


13% 


15% 


Child protective services 


2% 


12% 


54% 


32% 


Battered women 


21% 


34% 


18% 


26% 


Chemical dependency 


18% 


18% 


26% 


37% 


Personal assistance 


6% 


15% 


35% 


45% 


Translators 


22% 


12% 


14% 


52% 


Parent stipend 


13% 


2% 


17% 


68% 


Exhibit reads: Most projects provided transportation services: 42 percent of the projects provided 
transportation exclusively through Even Start resources, 41 percent through Even Start and cooperating 
agency resources, and 14 percent exclusively through cooperating agency resources. 



special activities were worth reporting. Exhibit 5.9, however, gives a sense of the range 
of special activities offered by projects. 

More than one-half of the projects (57 percent) take participants on field trips. Common 
destinations included libraries, zoos, museums, historical landmarks, farms, and 
businesses. Field trips are a way to provide common experiences to encourage learning 
and parent-child communication and interaction-they usually had an educational 
component and promoted social interaction. 
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Exhibit 5.9 




Percentage of Projects Reporting Types of Special Even Start Activities 

{1991-92 Program Year) 


Activity 


Percent of Projects 


Field trips 


57% 


Parties / picnics / meals 


45% 


Holiday/seasonal activities 


40% 


Ongoing recreation / socialization activities 


28% 


Literacy theme events 


26% 


Arts / cultural / ethnic activities 


21% 


Graduation / completion / recognition ceremonies 


19% 


Community awareness 


18% 


Special projects 


17% 


Workshops / training / speakers 


10% 


School visits / programs 


9% 


Open house / fair 


7% 


Social services 


6% 


Exhibit reads: Fifty-seven percent of all projects conducted field trips as a special activity. 



Forty-five percent of the projects organized parties, picnics, and dining (e.g., family 
potluck meals) activities. These events help establish rapport and trust between the staff 
and participants, celebrate accomplishments of participants, provide opportunities for 
adults and children to be together, give adults a chance to support one another, and help 
motivate them to remain in the program. 

Holiday and seasonal activities were offered by 40 percent of the pojects. They provide 
the opportunity for projects to plan educational experiences uniquely associated with 
special holidays (e.g., Thanksgiving, Christmas, Easter, Martin Luther King's birthday) and 
integrate them into social occasions. 

Events built around literary themes (e.g., day long literary fairs) were reported by 26 
percent of the projects. Finally, many types of special events were reported by less than 
one-quarter of the projects, including arts/cultural/ethnic activities, ceremonies to 
recognize graduation or completion of requirements, community awareness projects, 
special projects (e.g., plays, toy making, letter writing, bake sales, puppet shows), 
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workshops and speeches, school visits, open houses, and assorted social services {e.g., 
offering banking services to participants, having children fingerprinted). 

Many projects (28 percent) organized ongoing recreational and socialization activities for 
families. Examples include nature walks, going to movies, swimming parties, and 
attending circus performances. Recreational and leisure activities provide a break from 
routines and give families the opportunity to be together in a moderately controlled 
environment, often less stressful than the home. Recreational and leisure activities also 
offer a nonthreatening context in which to practice newly acquired skills and develop 
skills in dealing with other people. 



Cooperating Agencies and Cooperative Arrangements 

Even Start projects are required to establish cooperative arrangements with other agencies 
to avoid duplicating services. This strategy provides for optimal use of limited resources 
and allows projects to concentrate resources to fill service gaps. Each project reported 
on the cooperative arrangements it established to provide core and support services 
during the 1991-92 program year. 



Cooperative Arrangements for Core Services 

Even Start projects were involved in 2,808 cooperative arrangements to provide core 
services during the 1991-92 program year (Exhibit 5.10). This represents an average of 
27 cooperative arrangements per project. Fifty percent of the arrangements were for 
parenting education, 24 percent for adult education, and 26 percent for early childhood 
education. 1 

A wide variety of organizations cooperated with Even Start projects. The most common 
was "other departments and pr grams within the public schools" which accounted for 25 
percent of the parenting educacion arrangements, 23 percent of the adult basic education 
arrangements, and 35 percent of the early childhood education arrangements. The next 
most common type of cooperating agency was "local, county, state, or tribal agencies 
or organizations" which accounted for 24 percent of parenting education arrangements, 
20 percent of adult basic education arrangements, and 1 5 percent of early childhood 



1 While half of the collaborative arrangements made by Even Start projects are for parenting 
education, we noted earlier in this report that parenting education is the core service most often 
provided by Even Start staff. This apparent contradiction is explained by the fact that early childhood 
education and adult education services are generally available from a relatively small number of existing 
providers, and Even Start grantees seldom provide these services with thei' own staff. On the other 
hand, parenting education covers a wide range of topics and services rarely exist in any organized 
fashion. Hence, Even Start grantees often provide or coordinate the provision of these services using 
their own staff. They enter into multiple cooperative arrangements for parenting education services 
because no single agency is able to provide this service. 
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Exhibit 5.10 










Number and Percentage of Arrangements to Provide Core Services by Type of Organization 

(1991-92 Program Year) 


Type of Organization 


Parenting 
Education 


Adult 
Education 


Early Childhood 
Education 


Total 




N 


II % 


N 


II ™ 


N 


II % 




Other departments/programs 
wiinin puDiic scnoois 


345 


25% 


155 


23% 


254 


35% 


754 


Post secondary: college, 
university, trade-technicai 
school or institute 


128 


9% 


121 


18% 


53 


7% 


302 


Head Start or Home Start 


99 


7% 


24 


4% 


81 


11% 


204 


Day care or preschool 
programs 


41 


3% 


13 


2% 


66 


9% 


120 


Local, county, state or tribal 
agencies or organizations 


336 


24% 


137 


20% 


108 


15% 


581 


Foundations, fraternal groups 


93 


7% 


43 


6% 


39 


5% 


175 


Volunteer groups 


88 


6% 


64 


10% 


45 


6% 


197 


Other community-based 
organizations 


151 


11% 


70 


10% 


51 


7% 


272 


Church, temple or mosque 


57 


4% 


18 


3% 


12 


2% 


87 


Other 


67 


5% 


26 


4% 


23 


3% 


116 


Total 1 ,405 


100% 


671 


100% 


732 


100% 


2,808 


Exhibit reads: Twenty-five percent of all 1,405 cooperating arrangements to provide parenting education 
with other departments/programs within public schools. 


were 



education arrangements. Other cooperating agencies included postsecondary institutions, 
Head Start, day care or other preschool programs, foundations, volunteer groups, and 
other community-based organizations. Although they were mentioned infrequently, 
religious institutions (church, temple, or mosque) were involved in more than 80 
cooperative arrangements. 

Exhibit 5.1 1 displays the percentage of cooperative arrangements by core service area 
and source of authority over activities* The locus of authority for activities is evenly split 
between Even Start (about 33 percent of the agreements), the cooperating agency (about 
38 percent of the agreements), and joint decision-making (about 29 percent of the 
agreements). 
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Exhibit 5.11 






Percentage of Cooperative Arrangements to Provide Core Services 
by Core Service Area and Source of Authority Over Activities 
(1991-92 Program Year) 


Source of Authority Over 
Even Start Activities 


Parenting 
Education 


Adult 
Education 


Early Childhood 
Education 


Even Start 


34% 


34% 


30% 


Cooperating Agency 


36% 


35% 


42% 


Co-Deciding 


30% 


30% 


27% 


Exhibit reads: Thirty-four percent of ail cooperative arrangements to provide parenting education were 
governed by Even Start. 



Exhibit 5.1 2 shows the types of mechanisms used to govern Even Start activities in each 
core service area. It reveals that decision making arrangements may involve any of the 
means listed, but that informal agreements are used more often than any other type of 
coordination (in 45 percent of the cases), with no differences across core service areas. 
Three other types of decision making are used with roughly the same frequency: informal 
communication (17 percent), informal advisory groups (15 percent), and formal written 
agreements (1 7 percent). Joint boards are used least often (in five percent of the cases). 



Cooperative Arrangements for Support Services 

Support services enable families to participate in Even Start core services by removing 
barriers to their participation. The support services most commonly provided through 
collaborative arrangements were transportation, meals, health care, counseling and child 
care. Exhibit 5.13 shows that projects engaged in 1,176 cooperative arrangements for 
support services, and displays the number and percentage of projects providing a 
particular support service through a cooperative arrangement. 



Implementation Problems and Solutions 

Two types of implementation issues are addressed in this section. First, each project was 
asked an open-ended question about major barriers to the implementation of Even Start 
as well as the strategies or solutions used to deal with the barriers. Second, projects 
were asked to identify features of the Even Start law or regulations which they felt 
needed revision to permit more effective implementation. 



Even Start 



5-13 r 



Characteristics of 
Even Start Projects 



Exhibit 5.12 



Percentage of Cooperative Arrangements to Provide Core Service by 
Core Service Area and the Means of Reaching Decisions About Activities 

(1991-92 Program Year) 



Means of Decision Making 



Parenting 
Education 



Adult 
Education 



Early Childhood 
Education 



Informal communication 
Informal agreement 
Informal advisory group 
Formal written agreement 
Joint board 
Other 



16% 
46% 
16% 
16% 

3% 
0% 



18% 
44% 
14% 
18% 

5% 
0% 



17% 

45% 

12% 

16% 

8% 

0% 



Exhibit reads: Sixteen percent of all cooperative arrangements in parenting education relied on informal 
communication as a means of reaching decisions about activities. 
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Exhibit 5.13 

Cooperative Arrangements for Support Services 
(1991-92 Program Year) 


Number of Cooperative 
Support Service ! Arrangements 


Percent of Projects 


Transportation 


139 


68% 


Meals 


105 


59% 


Health care 


105 


59% 


Counseling 


143 


58% 


Childcare 


109 


51% 


Other 


156 


47% 


Nutrition 


63 


42% 


Advocacy 


93 


38% 


Employment referrals 


51 


32% 


Child protective services 


33 


26% 


Referrals for battered women 


39 


24% 


Personal services 


38 


19% 


Chemical dependency referrals 


29 


14% 


Parent stipend 


16 


13% 


Mental health 


30 


12% 


Translators 


27 


11% 


Total 


1,176 


100% 


Exhibit reads: Sixty-eight percent of all reporting projects used cooperative arrangements to provide 
transportation. 
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Barriers to Program Implementation 

Many different types of barriers were identified by Even Start projects for the 1991-92 
program year (Exhibit 5.14). The most common barriers were difficulties in the 
recruitment, retention, attendance, and motivation of families {41 projects across both 
cohorts), problems of communication and coordination with cooperating agencies (28 
projects), financial problems (20 projects), problems with the evaluation (15 projects), 
staffing problems (16 projects), and problems with facilities and space (16 projects). 
Other problem areas included the difficult social service needs of the family and 
community (12 projects), a lack of quality child care (11 projects), program regulations 
and limitations imposed by the local Even Start model (10 projects), a lack of local 
expertise and materials (7 projects), scheduling difficulties (8 projects), problems 
communicating with families (8 projects), a lack of locally available child care services (8 
projects), and a lack of transportation for families (6 projects) or staff (6 projects). 



Features of the Law or Regulations that Would Enhance Implementation 



Exhibit 5.15 summarizes the responses given when projects were asked about features 
of the Even Start law or regulations that co^!u be revised in order to enhance program 
implementation. A companion exhibit (Exhibit 5.16) lists specific comments from 
projects. The responses are paraphrased to give the reader a quick sense of what, in 
some cases, were fairly detailed points that projects wished to make. 

During 1991-92, 22 projects responding to this item indicated a need to make eligibility 
criteria more flexible. Twelve projects felt that the law should allow more flexible 
program design; 11 projects raised general concerns about the evaluation; ten raised 
specific issues related to the national evaluation; and others expressed concerns about 
fiscal matters (13 projects); state control (three projects); and administrative concerns 
(one project). Many of the concerns expressed by projects could be addressed without 
making changes in the law or regulations, but no attempt to limit responses has been 
made here. It is worth noting that the number of projects expressing concerns in most 
areas has dropped from 1990-91 to 1991-92. For example, concerns about eligibility 
criteria were noted by 57 projects in 1990-91 , but only 22 projects in 1991-92. A similar 
pattern can be seen for all areas. 



Technical Assistance Needs 

AH projects were asked to identify the areas in which they would like technical assistance. 
Because the item was open-ended, the frequency with which any one area was identified 
should be interpreted with caution. A priority listing would be better, giving all projects 
the chance to consider each potential area of need. Nonetheless, responses of projects 
to the open-ended item gives a rough indication of Ihe prevalence of certain needs. 
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Exhibit 5.14 

Project Implementation Barriers and Resolutions 
(1991-92 Program Year) 


Barriers 


Resolutions 


Problems of Recruitment, Retention, 
Motivation, and Attendance of 
Participants 

Number of references: 55 
Number of projects: 42 


Wrote a proposal to handle these duties for 5 hours per week 

Allowed one day vacation period right before holidays 

Provided awards, family trips, awareness of what commitment <s 

Used more literacy materials geared to male interests 

Provided home-based adult education when necessary 

Home-based instruction for those with difficulties scheduling classes 

Call before visit or visit to set up time for home visit, bring small gift 

Make Even Start an inviting environment; families are to call if they 

must miss; we visit those who miss 

Field trips have helped, but not a complete solution. 

Changed schedule to avoid conflict with migrant programs 

New volunteer-in-school program to build familiarity with school/staff 

We accept families for all three core services only 

Hired new male staff, one to start male support group 

Offer broader range of training and emphasize hands-on activities 

Family fun fair for recruitment — 153 attended, enrollment way up. 

Calls made ahead when possible; we just kept returning to the home 

Remind contacts frequently of services available 

Even Start students tell others about program and their success. 

Opening of center and improved recruitment in Head Start 

Concentrated services during a shorter time 

Cannot miss more than three events or they are out of program 

Getting more firm on requirements, flexible in when to start and break 

Give incentives, i.e., coupons for fast food restaurants 

Present parent-child activities before adult education classes; give 

sessions more structure 

Using Even Start staff from a school's faculty helps enrollment 

Better screening so they know this is more than adult education 

"Bonus bucks" awards to participants; monthly recognition events 

Money reward for attendance 

Develop a trusting relationship 

More frequent/extensive home visits 

Giving out books helps; use volunteer tutors 

Family open house change attitudes of man, allowing wife to attend 

Be non-judgmental, supportive, accepting, build self-esteem 

Developed alternative places for visits, i.e., library, school sites 

Staff attempts to involve entire family 

Invited husbands to attend class and parties, wives counseled 
Teachers stop by with "lesson packet" for missed lesson 
Opened ECE Family Center—improved recruitment and participation 
As JOBS is better-implemented locally, recruitment and participation 
should improve 

Follow-uD visits to individuals who left program || 
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Exhibit 5.14 
(continued) 

Project Implementation Barriers and Resolutions 
(1991-92 Program Year) 


ft Mr rt ore 


Resolutions 


Facilities/Space/Equipment 
Problems 

Number of references: 1 8 
Number of projects: 1 6 


Made childcare space a top priority 
Meetings at coordinator's home 

Exploring possibility of moving program to another school site 

Now have a big building with separate age-grouped rooms and a gross 

motor room 

Negotiating with city for a street sign (clients cannot find facility) 

Additional space is provided by school district 

Even Start given priority for portable classroom for 1 993 

Maintenance department making repairs/remodels 

Wants to have a classroom just for after-school component 

Rotate rooms as they are available 






Social Service Needs of 
Families/Community 

Number of references: 1 4 
Number of projects: 1 2 


Networlcino and mpptinn^ with afipnr*ipc HanHlinn hnncinn 

Collaborating with Family Mentor Program to provide counseling 
Make referrals 

Offer folks a "break" from Even Start 

Budget revision to add a part time speech pathologist 

Education and self-esteem building 

Refer to agencies for social needs 

Staff development/training to deal with "at risk" population 
Refer families to social services 

Locked doors at B'OO D.m. and moved ornaram tn nnp pnH nf huilHinn 

w»-»ww WWW! M Mfc WaWV • 1 * I • Ul 1 \J 1 1 IU ¥ Vw U^l will IU \Jt CI IU \J 1 L/UIIV-Jll IU 

Referred for further resources in counseling/setting priorities and goals 
Provided workshops and maintained a link with women's crisis center 
Limit home-based program to least active hours and not alone 


Lack of Quality of Childcare 

Number of references: 1 2 
Number of projects: 1 1 


Some now receive full time childcare through Project Independence 
Opened an approved child care center 
Hired a work-study student to help them with homework 
A preferred babysitters registry has been piloted this year 
Social Services is assisting with childcare and they are funding 
Offered to pay neighbor or relative to babysit, but few accepted 
Parents found own babysitters, and program reimbursed 
Cooperative agreement with DHS for childcare reimbursement 
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Exhibit 5.14 
(continued) 

Project Implementation Barriers and Resolutions 
(1991-92 Program Year) 


Barriers 


Resolutions 


Scheduling Difficulties with Families 

Number of references: 8 
Number of projects: 8 


Adjusted class schedules to two 1 Vz hour sessions (a.m. to p.m.) 
Individualized instruction, but concerned about budget ramifications 
Offer to keep in touch; some find they can continue 
Set up morning and evening classes 

Staff have flex hours to accommodate the different schedules 
Instituted an attendance policy: three unexcused misses per month 
New schedule of five hours per day, two days a week 


Communication with Families 

Number of references: 8 
Number of projects: 8 


Home visits, one-on-one interaction, easy-to-read notes sent home 
Home visitors leave notes; schedule visit a month ahead 
Notes sent home with elementary students 
Give families postcards to let us know when they will be home 


Lack of ECE/ABE/PE Services 
Locally 

Number of references: 8 
Number of projects: 8 


Advanced arrangements with Community College Adult Education 
Collaborated with local company in developing adult learning center 
Constant contact with collaborating agencies to develop new resources 
Emphasizing two and three year-olds for Even Start 
Implementing GED computer program and high school diploma program 


Transportation for Families 

Number of references: 7 
Number of projects: 6 


Partially resolved using buses 

Two, 1 2-passenger vans purchased and rotated to cover large area 
Increased communication with various day care centers 
Head Start Demonstration Project will provide funds for larger bus 
Home-based instruction for those with difficulties getting to center 
Using county van when available 

A day care program provides needed transportation (cooperative effort) 
Negotiated contract with school district 
Gave parenting workshops at each of the five satellite centers 
Meetings during the day at coordinator's house 

Added second v^n roirfp OAvinn milpant* tn Hriv/i nn narontc 
r~\\*\**+\* q^ijwiiu vaii i uuvCf f/ayiiiu limeade wj uiivill^ UdrciiLo 

A church donated space, so we can serve multiple families in that area 

Tried taxi service, but not reliable 

Various means: cabs, vendor, pay mileage, bus tickets 

Paying $.15/per mile 

Coordinating services with the schools' transportation department 

Reimbursing parents/volunteers at a mileage rate lessens difficulties 

Meetings planned with public school transportation director 

Contracting a van and driver 

Parents provide transportation in some cases 

Instituted carpools 

Located adult/child activities within walking distance of homes 
Mobile classroom: a used bookmobile was purchased and renovated 


Transportation for Staff 

Number of references: 7 
Number of projects: 6 


Included transportation costs in salary 
Classes meet at the Even Start site 

Increased frequency of maintenance prevents big problems 
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Exhibit 5.14 
(continued) 

Project Implementation Barriers and Resolutions 
(1991-92 Program Year) 


Barriers 


Resolutions 


Extreme Educational Needs of 
Parents 

Number of references: 2 
Number of projects: 1 


Home visits, attend school functions, students recruiting students, etc. 
Provide one-on-one instruction via trained adult literacy volunteers 





Exhibit 5.15 






Features of the Even Start Law or Regulations That May Need 
To Be Revised to Permit More Effective Implementation 
(1990-91 and 1991-92 Program Year) 






Number of Projects 


Features of the Law 




1990-91 


1991-92 


Eligibility criteria 




57 




22 


Program design 




33 




12 


General evaluation concerns 




20 




11 


NEIS forms/instruments 




15 




10 


Fiscal issues 




15 




13 


Concerns about state control 




0 




3 


Administrative concerns 




15 




1 


Exhibit reads: In 1990-91, 57 projects identified eligibility criteria as a feature of the law that needed to 
be revised. 
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Exhibit 5.16 

Abstracted Project Comments on Features of the Even Start Law 
or Regulations That May Warrant Revision 
(1991-92 Program Year) 


Features of the Law 


Commentary 


Eligibility Criteria 


Eliminate the requirement that families live in Chapter 1 areas. 

There is low income outside Chapter 1 schools. 

Modify requirements to allow for people who move. 

Poor and needy families live in other than Chapter 1 areas. 

The main barrier is Chapter 1 service areas. Needy people live 

outside of the Chapter 1 area. 

Need to serve families outside our local LEA; families move. 
What if a school becomes Chapter 1 in-eligible? Must families 
be dropped? 

Any parent, regardless of age, should be eligible for Even 
Start. 

Services should be provided for children up to the age of nine. 
Why does family literacy component have to be directed to 
preschool child only? 

Age limitation of 7 years prevents help to many. 

Change age to children younger than sixteen. Thirteen would 

benefit some areas. 

Even Start should be available before birth of first child to 
prevent FAS, etc. 

Extend age limit so ABE students can get parenting. 
Allow teen parents not in other programs to be eligible. 
Should work with all illiterate families regardless of ages. 
After achieving GED, may want to stop adult education. 
Allow people with GED to get trained for real work. 
Should not have to drop ABE when they receive GED. 
DSS referrals are "psychologically at-risk", but do not qualify. 
Clarify who will receive services when target child turns eight. 
Clarify who will receive services when target parent finishes. 
Definition of "need" requires a better explanation. 
Allow current migrant to stay in Even Start if they settle. 
Drop the requirement of current migrant status for migrant 
program participants. 
Eligibility criteria need to be more flexible. 
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Exhibit 5.16 
(continued) 

Abstracted Project Comments on Features of the Even Start Law 
or Regulations That May Warrant Revision 
(1991-92 Program \ear) 


Features of the Law 


Commentary 


Program Design 


Clarify regulations to allow dual use of classrooms for public 
school and Even Start. 

Eliminate NDN requirements in Even Stan legislation. 

Guidelines for core services need to be reconsidered. 

Rural programs need to be specially designed to accommodate 

transportation/childcare 

Improve guidelines (e.g., what constitutes participation?) 
Mandate Chapter I, 11, Even Start, Head Start work together. 
Regulations needed for other federal programs to provide 
collaboration with Even Stan. 

We need written guidelines to identify which government 

program will service what families. 

Allow for transition from GEO to higher education. 


General Evaluation Concerns 


Revise the testing requirements. 

A voluntary program needs a better way to gather data. 
Eliminate testing of parents and children. 
We need clarity on responsibility and extent of evaluation. 
Submit DQI forms six weeks prior to end of reporting period. 
We would like the DQI sent at a different time (sooner). 
DQI information needs to be handled more efficiently. 
Evaluation takes a third of staff time. The new Form 11 helped. 
We spent 378 hours in June 1992 for 187 families. 
Paperwork remains time-consuming and confusing. 
The paperwork in May is overwhelming. 
Reduce paperwork — it is too labor intensive. 
Sample participant information instead of every family. 


Concerns About State Control 


Concerned that funding will be administered unevenly. 
Concerned about law that hands programs to state level. 
Even Start state representative opposed to refunding anyone, 
thus must be mandated by Federal government. 
State will not allow us any funding after year four. 


Administrative Concerns 


Change in-kind requirements. 

Make Even Start funding coincide with local/state fiscal years. 
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The number and percentage of projects requesting each type of technical assistance are 
presented in Exhibit 5.17. The most frequent need identified by projects was for 
evaluation assistance, including the completion of the NEIS forms (16 percent). This is 
a substantial decrease from the percentage of projects that wanted assistance in 
evaluation during 1990-91. 

All three core service areas were identified by five percent of the projects as areas in need 
of technical assistance. Projects asked for information on effective parent-child 
interaction activities, selecting appropriate parenting curriculum materials for Even Start 
parents, getting parents involved in the education of their children, accessing ideas from 
research on parental involvement, and finding materials appropriate for home-based 
activities. Similar needs for materials and activities were identified for early childhood 
education. Most of the assistance needs in adult education were general (e.g., "the adult 
learner," adult education) suggesting more basic needs for assistance in that area. 

The remaining areas of assistance listed in Exhibit 5.17 are self-explanatory. Thirteen 
percent of the projects were interested in cross-project sharing of information to learn 
more about running responsive and effective Even Start programs; seven percent of the 
projects wanted assistance in staff development; six percent wanted help with funding, 
and five percent or fewer projects requested technical assistance in a variety of other 
areas. 
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Exhibit 5.17 

Areas of the Program for Which Technical Assistance Is Wanted 
(1991-92 Program Year) 


I Percent of Projects 


tvaiuation/Nhlb 


16% 




Cross project sharing 


13% 




Staff development 


7% 




Funding/fiscal issues 


6% 




Increasing participant involvement 


6% 




Curriculum materials 


6% 




Integrating components 


6% 




Program administration 


6% 




Parenting information 


5% 




Early childhood education 


5% 




Adult education 


5% 




Transition to state administration 


5% 




Home visits 


4% 




computer uses 


4% 




Interagency collaboration 


4% 




Local evaluation/assessment 


4% 




Recruiting participants 


3% 




Effective practices 


2% 




Social problems 


2% 




PEP/NDN 


2% 




Transportation 


1% 




Information on technical assistance available 


1% 




Exhibit reads: Sixteen percent of ell reporting projects requested technical assistance in 
svaluation/NEIS. 
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Section Six 

Participation in Even Start Core Services 



This section presents data that describe the length of time families participate in Even 
Start core services, reasons for exiting the program, the amount of service delivered, and 
information on service delivery modes. The data are drawn from the National Evaluation 
Information System (NEIS), and represent projects from Cohorts 1 and 2 as well as 
projects for migrant families. Data are combined across cohorts, except in instances 
where separating the cohorts illuminates findings. 



Background 

Although the law and regulations do not specify any set length or intensity of participation 
in Even Start, the program is intended to serve families which have low-literate adults and 
have children younger than eight years of age. It can take time to remedy literacy 
problems, and the law ar.d regulations allow projects periods of up to four years. This 
permits projects to provide relatively long-term, multi-year services. Moreover, the law 
requires grantees to encourage participants to remain in the program for a time sufficient 
to meet program goals. Some projects intentionally recruit families with very low-literate 
adults and plan to serve them for several years, while other projects plan to provide 
shorter-term services to families that have an adult who can reasonably expect to attain 
a GED within the coming year. 

The Department of Education's expectation is that all Even Start families will participate 
fully in the program throughout the period that they are enrolled. This means that each 
family should take part in each of Even Start's three core services during their 
involvement with the program. 

This evaluation has multiple measures of program participation. One measure comes from 
an annual interview with a family member, usually the mother, conducted by Even Start 
staff. On the interview form, project staff indicate whether, during the year, the family 
was an active participant in each of the three core services. This judgement on the part 
of the project staff member provides a binary measure of participation in each core service 
area for each family for each year of the evaluation. A second measure of program 
participation comes from monthly "contact logs" which are used by project staff to record 
the number of hours of participation each month by each family in each core service area , 
Contact log data are matched against the annual interview data, and a family is counted 
as participating during the year in a given core service either if there is one or more 
contact logs indicating that the family spent time in programmatic activities, or if the 
annual interview indicated that the family was an active participant. 
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The process described above yields a yes/no measure of participation in each program 
year rather than a measure of quantity of participation. However, multiple measures of 
the quantity of service received in Even Start are obtained from the contact log data. One 
such measure is a count of the number of months for which contact log data are 
submittdEt; this is used as an indicator of the length of participation in the program. A 
second measure derived from contact logs is the number of hours spent in each core 
service area; this is operationalized in variables such as the total number of hours spent 
in each core service area (during a child's or adult's tenure in Even Start), and the number 
of hours spent in each core service area on a monthly basis. 

Methods of Recruiting and Retaining Even Start 
Families 

Recruiting, retaining, and motivating families to participate in Even Start are important 
activities undertaken by each project. 



Recruitment Strategies 

Projects use a variety of recruitment strategies to inform parents about the program and 
encourage them to enroll. During the start-up of a project, when families are unfamiliar 
with Even Start, in-person recruitment is the most successful approach. Over 50 percent 
of projects use home visits to recruit families, which includes both door-to-door "cold 
calls," where staff fan out in neighborhoods and talk with parents of young children, as 
well as appointments made to follow up on inquiries. Once the project is more well- 
known in the community, home visits are a less frequent recruitment strategy. More 
commonly, families are referred by other families, the school system, or community 
agencies. 

Both project directors and parents indicate that adult education is often the "hook" that 
brings families into Even Start. Many adults are interested in getting their GED in order 
to find employment or get better jobs. For some mothers, the child in Even Start may be 
their youngest, and they are looking ahead to the time when the child enters public school 
and the mother has fewer child care needs. Other mothers have indicated that as their 
children get older and progress in school, they are faced with the dual challenges of not 
having the reading and math skills to be able to help children with school work and, at the 
same time, encouraging their children to stay in school when they themselves dropped 
out. As one Even Start mother explained, "When my child came home from school with 
a book that I could not read or understand, I knew it was time for me to go back to 
school." 

The early childhood component of Even Start also is attractive to parents. In some cases, 
the early childhood program is seen as providing the necessary child care to enable 
parents to attend adult education classes. Especially in communities with limited public 
preschool options, the Even Start early childhood component is a valued commodity. 

Participation in Even 
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Maintaining parent participation is a continual challenge for Even Start projects and most 
incorporate incentives of one kind or another to encourage families to participate. 
Contracts or rules for attendance are one type of retention strategy. Contracts help to 
clarify parents roles and responsibilities (e.g., turn off the television during home visits or 
work through activity kits with their children at home). Contracts also are used to identify 
a specified level of participation in certain activities, such as attending adult basic 
education classes a minimum of twice a week, participating in two parenting workshops 
a month, or volunteering in their child's classroom at least twice a month. 

Projects work hard to maintain participation levels and understand that ihey often have 
to make extra efforts to encourage reluctant parents. Project staff report that regular 
participation depends greatly on the ability of staff to "bond" with program participants, 
as well as the degree to which participants come to see Even Start as providing a useful 
support system. Some projects use participants to recruit and retain other participants. 
As one mother put it while trying to talk a friend into attending, "If i can do this, so can 
you. n 

If parents are not participating fully, project staff often make special home visits to 
determine whether families are still interested in the program. Some projects have interim 
strategies that they put in place before they terminate a family. For example, a family 
that does not attend regularly may temporarily lose transportation services. Staff indicate 
that attendance policies serve to reinforce the idea that keeping an appointment is an 
important life skill that parents need to learn. Further, while project staff are always 
reluctant to have famiiies drop out of the program, they recognize that it is necessary to 
know when families are no longer actively participating in order to offer that slot to other 
interested families. Letting parents know others are interested in the program sends the 
message that it is a limited and valued resource. 

Projects provide tangible rewards for participation including prizes such as books and fans 
(in warm-weather climates) as well as t-shirts and certificates for attendance. Some 
projects let parents accrue -credits" for good attendance, which they can use to make 
purchases from an Even Start store stocked with household necessities (e.g., toothpaste 
or detergent) or to participate in special outings and field trips. 

Other incentives are woven into program activities- Food is particularly succes^ . in 
enticing parents to attend activities. More than half of the projects indicate that they 
have potluck suppers, picnics, or simple meals for families. 



Number of Participating Families 

Participation in Even Start has been increasing over time. This is due both to the adc^tion 
of new projects and to the increased efficiency of projects over time. The number of 
families that participated in Even Start core services is shown in Exhibit 6.1 . This exhibit 
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Exhibit 6.1 : Number of Families Participating 
in Even Start Core Services, by Program Year and Cohort 




199*90 1960-91 1991.92 Total in Any Ymt 



Note: Does not include 1 991-92 statistics on Cohort 3 projects that are being collected by Pelavin Associates. 
Exhibit reads: Cohort 1 projects served 2,460 families in 1989-90, 



includes data from projects first funded in 1989 or 1990, but does not include data from 
projects first funded in later years. 

Cohort 1 projects began in the 1989-90 school year, and 2,460 families participated in 
some core service during that year. Participation in those same Cohort 1 projects grew 
to 4,790 families in 1990-91 (an increase of 95 percent), and to 5,567 families in 1991- 
92 (an increase of 126 percent over the first year). Thus, the same projects, with the 
same level of resources, were able to serve more than twice as many families in their 
third year of operations as in their first year. The rate of increase was less from the 
second to the third year of operations than from the first to the second, and we expect 
that the third year numbers are relatively close to "steady state" levels, i.e., if projects 
were to be funded at the same levels for additional years, we would not expect the 
number of participating families to change appreciably. 

The large increases in numbers of families served over time can be attributed to the 
projects' need to deal with normal implementation problems in the first year of program 
operations (e.g., time had to be spent defining the program, recruiting staff, setting up 
operations) and the general difficulty of starting up a new program. Once these problems 
had been solved, projects became more efficient with the extra time and resources being 
devoted to recruiting and serving additional families. 
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A similar pattern is seen for Cohort 2 projects. A total of 1 ,956 families were served in 
Cohort 2 projects during 1990-91 (their first year of operations), and 4,126 families (a 
1 1 1 percent increase) were served in 1991-92. 



Participation Rates in Core Service Areas 

The Department of Education expects that all Even Start families will participate in each 
of the three core service areas during their time in the program. Exhibit 6.2a shows the 
percentage of families that participated in each core service area during the three years 
of study. Almost all Even Start families had a child that participated in early childhood 
education during each year of thg evaluation: 90 percent participated in 1989-90, 97 
percent in 1990-91, and 98 percent in 1991-92. Participation rates for parenting 
education were a little lower; 88 percent of families had a participating adult in 1989-90, 
94 percent in 1990-91, and 93 percent in 1991-92. At the beginning of Even Start, 
participation rates were lowest for adult education (54 percent in 1989-90). The 
Department of Education and local projects have worked hard to increase participation in 
adult education to 79 percent in 1990-91 and again to 90 percent in 1991-92. The 
percentage of families that participated in all three core service areas also has increased 
over the three years of study, from 46 percent to 74 percent to 84 percent. 

There seems to be approximately 20 percent of Even Start families that participate in one, 
but not both f of the adult-focused core service areas. Exhibit 6.2b contains detailed data 
on the number and percentage of participants by cohort and by year of the study. The 
cumulative statistics at the bottom of the exhibit show that almost all families (98 
percent) participate either in adult education or in parenting education, but that only 80 
percent of all families participate in both adult education and parenting education. 



Project-Level Variation in Participation Rates 

Participation rates are not uniform across Even Start projects. And, in fact, the averages 
presented above mask great project-to-project variation. Exhibit 6.3a is a distribution of 
project-level participation rates in all three core services. The shape of the distribution 
shows that most projects are able to engage a large percentage of their families in all 
three core services: 90 percent or more of the families participated in all core services 
for 65 of the projects. It also shows that all families participated in each core service in 
1 5 projects, while less than 60 percem of the families participated in each core service 
area in 8 projects. 

Exhibits 6.3b, 6.3c and 6.3d are distributions of project-level participation rates for each 
of the three core service areas. The distributions look roughly similar in shape, however, 
it can be seen that projects are much more likely to have very high participation rates for 
early childhood education and parenting education than for adult education. 
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Exhibit 6.2a: Percent of Families Participating in Even 
Start Core Services, by Program Year 

^1 100 




Core Service Area 



Exhibits reads: 90 percent of Even Start families had a child who participated in early childhood education 
during the 1989-90 program year. 



Multi-Year Participation 

Although Even Start projects are funded for four years, very few families take part for 
that amount of time. This fits with reports from project directors who, in the early years 
of the evaluation, identified the recruitment, retention, and motivation of families as the 
most common barrier to effective program implementation. 

Exhibit 6.4 shows that 55 percent of the families that began Even Start in Cohort 1 
projects during the 1989-90 year participated only in that first year, 25 percent 
participated in both the first and second program years, and 20 percent participated in 
three years. A similar pattern seems to be emerging for Cohort 1 families who started 
their participation in Even Start during 1990-91 . On the other hand, Cohort 2 projects 
appear to be more successful at retaining families across years. Of all Cohort 2 families 
that began Even Start in 1990-91 , 28 percent participated for one year, and 72 percent 
continued into a second year. 

These percentages are informative, but they are limited in that they only identify a family 
as having participated or not in a given year. They tell us nothing about the amount of 
participation during that year. A more detailed look at length of participation is given by 
Exhibit 6.5 which draws on monthly contact log data to show the number of months of 
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Exhibit 6.2b Number and Percent off Families 
Participating in Even Start Core Services 



Core Service 


Cohort 1 


Cohort 2 


Migrant 


Total 


■ i 


% 


N 


% 


» II 


% 


N 

• 


% 


1989-90 Participation 


















Adult education 


1,325 


54% 










1,326 


54% 


Parenting education 


2,161 


88% 










2,166 


88% 


Early childhood education 


2,223 


90% 










2,223 


90% 


Adult education or parenting education 


2,345 


95% 










2,345 


95% 


Adult education and parenting education 


1,141 


46% 










1,141 


46% 


All core services 


1,127 


46% 










1.131 


46% 


Total families 


2,460 


100% 










2,460 


100% 


1990-91 Participation 


















Adult education 


3,730 


78% 


1,577 


81% 


18 


100% 


5,325 


79% 


Parenting education 


4,517 


94% 


1,796 


93% 


17 


94% 


6,330 


94% 


Early childhood education 


4,649 


97% 


1,884 


97% 


18 


100% 


6,551 


97% 


Adult education or parenting education 


4,727 


99% 


1,869 


96% 


18 


100% 


6,614 


98% 


Adult education and parenting education 


3,520 


73% 


1,504 


78% 


17 


94% 


5,041 


75% 


All core services 


3,476 


73% 


1,497 


77% 


17 


94% 


4,990 


74% 


Total families 


4,790 


100% 


1,938 


100% 


18 


100% 


6,746 


100% 


1991-92 Participation 


















Adult education 


4,980 


89% 


3,615 


91% 


113 


84% 


8,708 


90% 


Parenting education 


5,198 


93% 


3,714 


93% 


133 


99% 


9,045 


93% 


Early childhood education 


5,433 


98% 


3,908 


98% 


134 


100% 


9,475 


98% 


Adult education or parenting education 


5,442 


98% 


3,915 


98% 


134 


100% 


9,491 


98% 


Adult education and parenting education 


4,736 


85% 


3,414 


86% 


112 


84% 


8,262 


85% 


Ai! core services 


4,674 


84% 


3,372 


84% 


112 


84% 


8,158 


84% 


Total families 


5,567 


100% 


3,992 


100% 


134 


100% 


9,693 


100% 


1 989*92 Participation (Cumulative) 


















Adult education 


7,787 


82% 


4,097 


90% 


121 


86% 


12,005 


85% 


Parenting education 


8,884 


93% 


4,286 


95% 


139 


99% 


13,309 


94% 


Early childhood education 


9,142 


96% 


4.444 


98% 


140 


100% 


13,726 


97% 


Adutt education or parenting education 


9,313 


98% 


4,452 


98% 


140 


100% 


13,905 


98% 


Adult education and parenting education 


7,347 


77% 


3,923 


87% 


120 


86% 


1 1 ,390 


80% 


AH core services 


7,277 


77% 


3,885 


86% 


120 


86% 


11,282 


80% 


Total families 


9,508 


100% 


4,531 


100% 


140 


100% 


14,179 


100% 


Exhibit reads: 54 percent of ail Even Start families had an adult who participated in adult education during the 1989-90 program year. 
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Exhibit reads: In 15 projects, 100 percent of the families participated in all three core services. 



Exhibit 6.3b: Distribution of Participation Rates 
by Project for Adult Education (1989-92) 




•04* 70-79 ft04# 

Percent of Families Participating in Adult Education 



Exhibit reads: Less than 70 percent of the families participated in adult education for ten projects. 
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Exhibit 6.3c: Distribution of Participation 
Rates by Project for Parenting Education (1989-92) 
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Exhibit reads: In most projects more than 90 percent of the families participated in parenting education. 
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Exhibit 6.3d: Distribution of Participation 
Rates by Project for Early Childhood Education (1989-92) 
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Exhibit reads: In 58 projects, 100 percent of the families participated in early childhood education. 
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Exhibit 6.4 












Years of Participation by Cohort and Year of Intake 




Cohort 


Intake 
Year 


One Year 


Two Years 


Three 
Years 




N 


% 


- 

N 


! 

% 


N 


% 


Total 


Cohort 1 


1 989-90 


1,352 


55% 


641 


25% 


312 


20% 


2,460 




1990-91 


2,088 


57% 


1,175 


43% 






3,652 




1991-92 


3,387 


100% 










3,387 


Cohort 2 


1990-91 
1990-91 


545 
2,586 


28% 
100% 


1,239 


72% 






1,937 
2,586 


Exhibit reads: Among families entering Even Start at a Cohort 1 site during 1 989-90, 55 percent 
participated only in that program year while one quarter (25 percent) also participated during 1 990-91 
and one-fifth (20 percent) participated in all three program years. 
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Exhibit 6.5: Number of Months of 
Participation for Even Start Families Starting in 1990-91 



0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 

Months of Participation 



Exhibit reads: About half of Even Start families that began the program in 1 990-91 participated for six or fewer 
months. 
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participation over a 24-month period for Even Start families that began the program in the 
1990-91 year. The months of service do not have to be consecutive, so that a family 
who participates in core services during October, skips November and December, and 
participates again in January would have two months of participation. 

The exhibit shows that about one-quarter of the families (27 percent) that began the 
program in 1990-91 received some core services in three or fewer out of the total 
possible 24 months, about one-half of the families {52 percent) participated for six 
months or less, about two-thirds (67 percent) participated for nine months or less, and 
about four-fifths (79 percent) participated for 1 2 months or less. Conversely, 2 i percent 
participated for more than 12 months. 



Reasons for Exiting from Even Start 

There are many reasons for turnover of families in Even Start (see Exhibit 6.6). Some of 
these are positive, e.g., a parent found a job and moved out of the project's catchment 
area, and some are negative, e.g., the family lost interest or the program didn't meet their 
needs. A reason for leaving was reported for only about half of the families that left the 
program. Project staff are, in many cases, unable to track families as they depart. 
Families that exited the program for unknown reasons may be different in important ways 
from families that exited for a known reason. For example, families which leave for 
unknown reasons may be more likely to move, to be dissatisfied with the project, or to 
be difficult-to-reach families than families that leave for a specific reason. 

Based on families where there was a reason for leaving, completion of the planned 
educational program was listed for 27 percent of families that exited Even Start. This 
includes families where all members completed their educational program as well as 
families where parents obtained their GED or a full time job, at which time the entire 
family exited the program. 

Moving out of Even Start's catchment area was the most common reason for leaving the 
program, listed for 31 percent of families. This large percentage of movers raises the 
issue of whether project directors are aware of and are using their option to continue 
serving families that have moved but are close enough to continue participation (i.e., 
moved to another catchment area in the same school district). 

Fourteen percent of the families left Even Start because of a general lack of interest in the 
program and a subsequent refusal to participate. Another 14 percent had a family crisis 
of one sort or another that prevented them from participating. Seven percent left the 
program because they became ineligible due to a change in the family situation, i.e., there 
was no longer an eligible child or adult in the family. This could be due to federal or local 
eligibility requirements. Six percent gave a variety of reasons which suggested personal 
or structural conflicts or barriers to continued participation. These included medical 
reasons, work conflicts, pregnancy, scheduling conflicts, child care problems, and a lack 
of transportation. 
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Exhibit 6.6 






Reasons for Leaving Even Start 
(1991-92 Program Year) 




Reason for Lpavina 


w 


Percentage of 
Families Giving A 
Reason 


No reason given 


4,235 




Reason given 


4,369 


100% 


Completed planned educational program 


1,174 


97% 


Moved from area 


1,366 


31% 


Family crisis 


608 


14% 


Lack of interest, refused to participate 


618 


14% 


No longer eligible (federal or local 
requirements) 


321 


7% 


Conflicts, barriers to participation 


221 


6% 


Other 


61 


1% 


Exhibit reads: 27 percent of the families which left Even Start did so because they successfully 
completed their planned educational program. 



Amount of Core Services 

The contact logs compiled by Even Start staff record the amount of time spent in each 
core service area on a monthly basis. The mean amount of service for each core service 
area is substantially higher than the median, indicating that some families receive very 
large amounts of service, while many more received relatively small amounts of service. 
We present both medians and means in the following discussion; in this situation, the 
median is a better representation of the typical Even Start family since it is relatively 
unaffected by outlying values. 



Total Hours of Services Received 

The typical Even Start family received core services in approximately six different months 
during their enrollment in Even Start (Exhibit 6.7a). The total amount of service received 
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Exhibit 6.7a 
Measures of Amount of Core Services 




Percentiles 






Core Service 


25% 


50% 


75% 


Mean 


1 SD 






(Median) 






J 


Total hours served (1990-91 












cohort) 












Adult education 


14 


39 


107 


85.7 


121.8 


rarenung education 


10 


27 


59 


48.9 


76.5 


early childhood education 


17 


84 


287 


188.9 


246.4 


1 otai months served ( 1 990-9 1 












cohort) 












Adult education 


o 


tt 

w 


i n 
i \j 


6.4 


4.5 


Parenting education 


3 


6 


11 


6.8 


4.7 


Early childhood education 


3 


6 


10 


6.6 


4.4 


Average hours per month 












Adult education 


4 


8 


16 


12.6 


13.3 


Parenting education 


2 


4 


8 


6.5 


7.1 


Early childhood education 


4 


14 


37 


24.6 


26.5 


Exhibit reads: Adults entering Even Start in 1990-91 participated in an average of 85.7 total hours of 
adult education. The median adult participated in only 39 hours. The difference between the average 
and the median reflects the fact that while most individuals participated only a few hours, some 
received many hours of service. 



by the typical family during their enrollment in Even Start is a median of 39 hours for adult 
education (mean of 86), 27 hours for parenting education (mean of 49), and 84 hours for 
early childhood education (mean of 189). 

There is great variation in the total amount of core services received by Even Start 
families. Exhibit 6.7b is a distribution of total hours of adult education. It shows that 
more than half of the adults entering Even Start in 1990-91 received less than 50 hours 
of adult education, 1 7 percent received between 50 and 1 00 hours, 9 percent received 
between 100 and 150 hours, and so on. Small percentages of families received very 
large amounts of adult education, (e.g., 9 percent received more than 250 hours). 
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Exhibit 6.7b: Distribution of Total Hours of Adult 




Total Hours of Adult Education 



Exhibit reads: More than half {.57 percent) of the adults entering in 1 990-91 participated in less than 50 hours 
of adult education. 

Note: Based only on months in which some core services were received. 

A similar pattern can be seen for parenting education (Exhibit 6.7c) and for early 
childhood education (Exhibit 6.7d). Seventy percent of Even Start families received less 
than 50 total hours of parenting education, 18 percent received between 50 and 100 
hours, 6 percent received between 100 and 1 50 hours, and about 6 percent received over 
150 hours of parenting education. For early childhood education, 42 percent of the 
families received less than 50 total hours, 12 percent received between 50 and 100 
hours, and the remaining 46 percent received more than 100 total hours, ranging up to 
over 600 hours of instruction. 



Monthly Hours of Service Received 

The typical Even Start family received 8 hours of adult education (mean of 13 hours), 4 
hours of parenting education (mean of 7 hours), and 14 hours of early childhood 
education (mean of 25 hours) per month (Exhibit 6.8a). This could equate to a once-a- 
week adult education class for two hours per week and a once-a-week parenting 
education visit or class for one hour per week. However, the data for early childhood 
education need to be further analyzed in order to see how amount of service varies by age 
of child. 
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Exhibit 6.7c: Distribution of Total Hours of Parenting Education 
for a 1990-91 Cohort Through May 1992 



60 



< 



O«o|. 



s. 



20 ! 



an 



mmmm 
mmmm 



: -f" 



W7S 



■^:|oU::! ; ,:!ji ^ 



0-49 



50-99 100-149 150-199 200-248 

Total Hours of Parenting Education 



250+ 



Exhibit reads: More than two thirds of the adults entering in 1 990-91 participated in parenting education less 
than 50 hours. 



Exhibit 6.7d: Distribution of Total Hours of Early 
Childhood Education for a 1990-91 Cohort Through May 1992 
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Exhibit reads: More than two-thirds of the children entering in 1 990-9 1 participated in early childhood education 
less than 50 hours. 
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As was the case for total hours of core services, there is great project-to-project variation 
in the average monthly hours of each core service. Exhibits 6.8b, 6.8c, and 6.8d show 
project-level distributions of the average hours per month of service for adult education, 
parenting education, and early childhood education. Most projects provide an average of 
fewer than 15 hours per month of adult education, with small numbers of projects 
providing much higher amounts (Exhibit 6.8b). The same pattern is seen for parenting 
education-a large group of projects provide an average of nine or fewer hours of 
parenting education per month (Exhibit 6.8c). There seems to be even greater variation 
in the amount of early childhood education services received. The distribution of average 
amount of service received (Exhibit 6.8d) is fairly flat, except that a relatively large 
number of projects provide an average of more than 45 hours of early childhood education 
per month. 

The amount of early childhood education service is related to the age of the child being 
served (Exhibit 6.9). The typical Even Start child who is less than one year of age 
received 4 hours per month of early childhood education (mean of 9 hours). This amount 
increases to 5 hours per month (mean of 10) for one-year-olds, 9 hours per month (mean 
of 16) for two-year-olds, 14 hours (mean of 23) for three-year-olds, and 22 hours (mean 
of 32) for four-year-olds. The pattern breaks for five-year-old children, who receive fewer 
hours of early childhood education services through Even Start (15 hours per month, 
mean of 25) because they are entering the public schools and this evaluation does not 
count hours received in kindergarten or primary grades as attributable to Even Start 
services. 

An important question is how this amount of early childhood education service compares 
with the amount received by children participating in Head Start. On average, four-year- 
old Head Start children seem to receive about 50 percent more hours of early childhood 
education per month than their same-age Even Start counterparts. To understand why 
this is so, it is important to realize that the Even Start averages reported above represent 
the amount of service actually received by participating children, not the amount planned 
or the amount offered. It also should be understood that Even Start does not prescribe 
a fixed amount of services to be delivered to each family. Some projects are highin 
nature, delivering large amounts of services to relatively small numbers of Head Start 
programs are more uniform in nature than Even Start projects. Even Start families, while 
others are low-intensity in nature, delivering lesser amounts of services to larger numbers 
of families. 

Head Start programs are more uniform in nature than Even Start projects. Even Start 
projects often serve children at a wide age range and may serve children at any one age 
through collaboration with several different preschool programs. In contrast, Head Start 
projects typically serve four-year-old children for a single year, using a five-day program 
for two and one-half or three hours a day-a total of 12 to 15 hours per week, or 48 to 
60 hours per month. However, this is the amount of service that is offered, not the 
amount actually received. Layzer and her colleagues (1993) provide data which show 
that Head Start children have an average 79 percent attendance rate. Making this 
correction lowers the monthly number of hours of early childhood education received by 
Head Start children to about 38 to 48 hours per month. This Head Start mean is about 
50 percent higher than the Even Start mean of 32 hours per month for four-year-old 
children. 
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Exhibit 6.8a: Median and Mean Hours Per Month 



of Even Start Core Services 




Core Service Area 



Exhibits reads: The typical adult participated in adult education services for 8.0 hours per month. 



Exhibit 6.8b: Project Level Distribution of Average 
Hours Per Month of Instruction for Adult Education 



<1 




Mean Adult Education Hours per Month 



Exhibit reads: Adults participated an average of four hours or less per month of adult education in 27 projects. 
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Exhibit 6.8c: Project Level Distribution of Average Hours 
Per Month of Instruction for Parenting Education 
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Exhibit reads: Adults in most projects participated in nine or fewer hours per month of parenting education. 



Exhibit 6.8d: Project Level Distribution of Average Hours 
Per Month of Instruction for Early Childhood Education 
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Exhibit reads: Children from 17 projects participated in early childhood education more than 45 hours per month. 
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Exhibit 6,9: Median and Mean Hours Per Month of 
Early Childhood Education, by Age of Child 
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Exhibit reads: The typical four-year old child participated in early childhood education for 22 hours per month. 



Even Start projects are required to provide some core services in a home-based setting 
and to provide some core services to adults and children together. The extent to which 
projects use these two "delivery modes" is the subject of this section. 

For the purpose of understanding how projects coped with these requirements, each Even 
Start project was categorized according to the amount of home-based services it provided 
in each core service area: 

Never Home-Based: A project was classified "never home-based" for a given core 
service area if zero percent of its service hours in that area were provided in a home- 
based setting. 

Rarely Home-Based: A project was classified "rarely home-based" for a given core 
service area if between one and 33 percent of its service hours in that area were 
provided in a home-based setting. 

Often Home-Based: A project was classified "often home-based" for a given core 
service area if between 34 and 67 percent of its service hours in thav area were 
provided in a home-based setting. 



Service Delivery Modes 
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Mostly Home-Based: A project was classified "mostly home-based" for a given core 
service area if between 68 and 100 percent of its service hours in that area were 
provided in a home-based setting, 

Exhibit 6.10 shows first that the provision of home-based services does not occur on a 
large-scale basis. Only 22 percent (17 + 5) of all projects provide more than one-third of 
their parenting education r a home-based setting, only 18 percent (9.5 + 8.5) provide 
more than one-third of thuir adult education in a home-based setting, and only eight 
percent (6 + 2) provide more than one-third of their early childhood education in a home- 
based setting. The exhibit further shows that Even Start projects are more likely to 
provide parenting education in a home-based setting than early childhood education, while 
adult education is the least likely core service to be provided in the home. Only 16 
percent of the projects never provided parenting education in the home, compared with 
24 percent of the projects that never provided early childhood education in the home, and 
35 percent that never provided adult education in uie home. 

Projects also were classified with respect to the number of hours that they provided in 
each core service area to adults and children together: never together (zero percent), 
rarely together (one to 33 percent), often together (34 to 67 percent), and mostly 
together (68 to 100 percent) (Exhibit 6.11 J 1 . As was the case for the home-based 
delivery mode, parenting education seems the most likely core service to be offered to 
parents and children together. All projects provided some parenting education to adults 
and children together, and 89 percent of the projects provided more than one-third of 
parenting education using the adult/child together mode. On the other hand, nine percent 
of the projects never provided early childhood education to adults and children together, 
and 43 percent never provided adult education to adults and children together. 



'it is not possible to aggregate the amount of core services that parents received with their 
children across core service areas because projects were allowed to duplicate hours across the three 
core services. 
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, Exhibit 6.10: Percent of Projects Using Different Amounts 
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Core Sendee Area 

Exhibit reads: 35 percent of the projects never delivered aduft education in the home. 



Exhibit 6.1 1 : Percent of Projects Using Different Amounts of Adult/Child 




Core Service Area 



Exhibit reads: 39 percent of the projects provided over two-thirds of parenting education services to adutts and 
children together- 
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Section Seven 



Approach to Assessing Short-Tew Effects 
on Even Start Participants 



This section presents our approach to assessing short-term effects of Even Start on 
children, parents, and families. Subsequent sections present data and describe our 
conclusions about short-term effects. The findings are important and we believe that they 
validly represent the effects of Even Start. However, they should be viewed as tentative 
because additional data are being collected (data from the 1 992-93 program year for the 
NEIS and a second round of follow-up data for In-Depth Study sites) which will be 
included in the final report from this evaluation. 

The analyses presented in this report focus on cross-project findings, although some 
subgroup analyses have been conducted. Additional analyses of effects for subgroups 
of projects (e.g., high vs. tow intensity projects) will be conducted for the final report. 



Areas of Effectiveness 

The conceptual model shown earlier in this report identifies the following major areas in 
which Even Start, hopes to produce effects: (1) effects on children, (2) effects on the 
literacy levels of parents, (3) effects on parenting and personal skills, and (4) effects on 
families. Exhibit 7.1 expands on the types of effects expected from Even Start 
participation and on the hypothesized relationships among types of effects. As has been 
discussed throughout this report, Even Start provides three core services (early childhood 
education, adult education, parenting education) and a set of support services. Increased 
service receipt is expected to lead to a series of short-term effects on children, parents, 
and families. Longer-term effects are hypothesized to result from continued program 
participation and from interactions among the short-term effects. This evaluation has 
focused on measuring the receipt of services for program participants (process outcomes) 
as well as short-term effects on children, parents and families. The evaluation has not 
been designed to assess longer-term effects-that will need to be the topic of a future 
study. 



Process Outcomes 

Implementation of an Even Start project should lead to an increase in the amount of 
services received in each core service area. Appropriate support cervices also should be 
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provided. While information on the types and quantities of services provided to Even 
Start participants was presented earlier in this report, a key issue for attributing any 
observed pretest-posttest gains to Even Start as opposed to other factors is the extent 
to which Even Start has been able to achieve its hypothesized process outcomes, i.e., 
increased receipt of early childhood education on the part ot children, and increased 
participation in adult education and in parenting education on the part of parents. These 
questions are difficult to answer, but some data on participation rates for children and 
adults from disadvantaged families in adult education, parenting education, and early 
childhood education programs are c vaiiable from this evaluation and from other related 
studies. 



Adult Education 

Even Start has greatly increased the participation of parents in adult education programs. 
Evidence to support this conclusion comes from the NEIS intake interview, where Even 
Start parents were asked whether they had previously participated in any of several 
different types of adult education (adult basic education, adult secondary education, GED 
preparation, or English as a second language) . In total, 29 percent indicated that they had 
taken part in some form of adult education prior to joining Even Start. Similar questions 
were asked in the In-Depth Study, where 40 percent of the parents responded that they 
had participated either in adult basic education, GED preparation, or English as a second 
language programs during the past year. The higher pre-Even Start participation rate for 
the In-Depth Study group may occur because the In-Depth Study sites overrepresent 
Hispanics, who are more likely than English-speakers to enroll in English as a second 
language programs. In any case, data from the NEIS show that in the third year of 
program operations, 90 percent of Even Start adults participated in adult education, 1 



Parenting Education 

Parenting education provided through Even Start was used much more often than in the 
absence of the program. Parents were asked questions about previous participation in 
parenting education programs as part of the In-Depth Study but not as part of the NEIS. 
Eight percent of the parents in the taOepth Study indicated that they had taken part in 
a parenting education program. This seems to be a reasonable estimate, since parenting 
education programs are rare. Data from the NEIS show that in the third year of program 
operations, 93 percent of Even Start adults participated in parenting education. 



families had to participate in at least one core service In order to be counted as a program 
participant. We know that some families were recruited, but never participated; and these were not 
counted. 
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Early Childhood Education 



We believe that Even Start has increased the percentage of children participating in early 
childhood education programs. However, determining the extent to which children would 
participate in early childhood education programs in the absence of Even Start is difficult, 
because Even Start serves children from birth to age eight, and early childhood education 
participation rates differ by age. In spite of this problem, Even Start parents were asked 
about the prior formal educational experiences of their child. Their responses (across 
children of all ages) were that 64 percent of children in Even Start had no prior formal 
educational experience, 14 percent had participated in Head Start, 13 percent took part 
in some other preschool, 12 percent were in kindergarten, and 5 percent were in primary 
school. 

We cannot disaggregate the Even Start data by age of child, and so data from the national 
longitudinal study of Chapter 1 were used to provide additional information on this issue. 
According to Puma et al. (1993), 26 percent of Chapter 1 students had participated in 
Head Start, and 35 percent had participated in aome other preschool, a total of 61 
percent. It makes sense that these percentages are higher than those reported by Even 
Start parents, because the Chapter 1 data were reported for first grade children, whose 
parents were able to reflect on their child's entire preschool experience. On the other 
hand, Even Start parents' reports of preschool experiences were limited by the age of the 
child— if a child was three years old at entry to Even Start, that child could not have 
participated in Head Start. Because of this problem, we feel that the Chapter 1 estimate 
of preschool participation is a better estimate of the typical early childhood education 
participation rate for a disadvantaged population than the statistics provided by Even Start 
parents. 

A recent analysis by the U.S. General Accounting Office (1 993) relies on Census data and 
provides estimates of preschool participation for poor and nonpoor three- and four-year- 
old children. The GAO concluded that 22 percent of three-year-old children and 49 
percent of four-year-old children from poor families had participated in preschool. The 
percentages were higher (33 percent and 57 percent, respectively) for children from 
nonpoor families. 

The NEIS data set tells us that by the third year of program operations, 98 percent of 
Even Start children participated in early childhood education, and so it is clear that 
regardless of the basis of comparison. Even Start has increased the use of early childhood 
education programs. 

To summarize, we believe that Even Start has achieved its hoped-for process outcomes. 
It has substantially increased participation rates in the three core service areas, from 
somewhere in the 30 to 40 percent range to 90 percent for adult education, from about 
eight percent to 93 percent for parenting education, and from 40-60 percent to 98 
percent for early childhood education. 



Even St art 
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Short-Term Effects 



Successful achievement of process outcomes should lead to short-term effects for 
children, parents, and families. Assessing short-term effects is an important focus of this 
evaluation. 

Children 

Even Start provides early childhood education services in order to have the short-term 
effects of increasing children's school readiness and literacy-related skills. We expect that 
short-term effects on children will be influenced by the provision of parenting education 
to adults in Even Start households. Measures of short-term effects on children include 
the PreSchool Inventory, the Peabody Picture Vocabulary Test, and the Child's Emergent 
Literacy Test. We expect to see a positive relationship between short-term gains on the 
PSI or PPVT and longer-term effects on school achievement. 



Parents: Literacy and Education 

Even Start provides two sets of services for parents: adult education and parenting 
education. Short-term effects on parent's literacy and education resulting from these 
services include improved functional literacy as measured by the Comprehensive Adult 
Student Assessment System, increased rates of obtaining a GED, and increased use of 
reading and writing in the home. We expect that there is a positive relationship among 
some of the short-term parent outcomes, for example, increases in functional literacy 
should lead to achievement of a GED. 



Parents: Parenting and Personal Skills 

Services for parents also are hypothesized to lead to a series of effects on parenting and 
personal skills. Measures in this area include improved personal skills (depression, 
mastery), an improved home learning environment, increased parental expectations for 
children, and increased teaching skills on the part of the parent. 



Families 

Achievement of short-term effects *or parents, along with the provision of support 
services, is hypothesized to result in a series of family-level effects including improved 
social support networks for the family, an improved ability of the family to access needed 
resources, and a general increase in literacy skills. 



Even Start 
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Long-Term Effects 



The theory underlying Even Start is that combining services for children with services to 
adults in a single, unified program will lead to long-term effects that are improved and 
more lasting than effects from more traditional, and separate, early childhood or adult 
education programs. Measuring these long-term effects is outside the scope of this 
evaluation, but follow-up studies ought to be designed to assess this important objective 
of Even Start. 



Children 

Many studies have shown that early childhood education programs can produce short- 
term effects on children's school readiness (Layzer et al., 1990). Studies also have 
shown that these effects "fade out" over time, so that no differences are observed past 
the early elementary grades {Lazar et al., 1977), although recent research has disputed 
the reason for fade out of effects (Barnett, 1993). Even Start hopes to change this 
pattern by combining increased school readiness skills for children with improved adult 
parenting and literacy skills, increased levels of community and school involvement, and 
better family resources. As a package, these short-term effects ought to lead to improved 
chances for long-term effects for children as measured by "school achievement.- Specific 
measures might include teacher ratings of behaviors, socialization, and achievement; 
grades; placement in Chapter 1 , special education, or remedial programs; grade retention; 
and many others. 



Families 

Improvements in adult literacy and parenting skills along with increased family resources 
and supports are hoped to lead to the longer-term effects on the economic well-being and 
stability of the family. Measures in this area could include reduced unemployment, 
increased earnings, and decreased welfare dependency, as well as continued higher levels 
of literacy for the family. 



Approach to Assessing Program Effects 

The analyses presented in this section rely both on the National Evaluation information 
System (NEIS) which provides longitudinal and cross-sectional data on all Even Start 
projects and participants, and on the In-Depth Study which provides longitudinal data on 
selected projects which agreed to implement randomized experimental evaluations to 
assess the effects of Even Start. The design of each of these parts of the evaluation is 
presented earlier in this report and in preceding reports from this evaluation (St. Pierre et 
al, 1991, 1993). 

As has been discussed above, Even Start hopes to have positive effects in several 
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different areas. Before going further, we need a definition of what is meant by "the effect 
of Even Start." The effect of Even Start on a participating family is the difference 
between an observation taken after participation in the program and what would have 
been observed if the family had not been in the program. Since it is impossible to know 
how a participating family would have performed if they had not been part of the 
program, we must estimate what that performance would have been. Such an estimate 
is called a "no treatment expectation" and is often generated by measuring control group 
families that are statistically equivalent to the program families. Post-program 
observations made on program families are then compared to the no-treatment 
expectation to yield a measure of program effect. 

The challenge in creating a no-treatment expectation is to ensure that the program group 
and the control group are, indeed, equivalent. To this end, the best control group is one 
that has been constructed by randomly assigning potential participants to Even Start or 
to a control group. This method ensures that the characteristics of participating families, 
on average, will not differ systematically from the characteristics of non-participants. 
Random assignment of families to Even Start or to a control group was used in designing 
the In-Depth Study. Five projects implemented a randomized design with the assignment 
of families conducted centrally by Abt staff. Details about the random assignment 
process were included in the Second Interim Report from this evaluation. Pretest-posttest 
data from these five sites are included in the analyses presented in subsequent sections 
of this report. 

The demographic characteristics at pretest of the families in the random assignment sites 
of In-Depth Study are presented in Exhibit 7.2. While the Even Start and control families 
are quite similar (none of the differences shown in Exhibit 7.2 are statistically significant), 
this sample of families differs somewhat from Even Stan families described on the NEIS. 
For example, 22 percent of the NEIS sample of adults speak Spanish as their primary 
language, compared with 36 percent of the Even Start families and 45 percent of the 
control families in the In-Depth Study. Similarly, about 18 percent of the families in the 
NEIS ate Hispanic, compared with nearly half of the families in the In-Depth Study. 

The sample size in each of the five In-Depth Study projects is small (there are a total of 
about 100 families in the Even Start group and 100 families in the control group), so data 
were pooled across projects for analytic purposes. Even Start children and their parents 
were pretested in the fall of 1 991 , upon entry to the program. Control group children and 
their parents were pretested during the same time frame. Posttesting for both groups s 
was done during the summer of 1 992. 2 Pretests and posttests were conducted by 
independent data collectors hired and trained by Abt Associates. 

Effects estimates for the In-Depth Study were based on a regression model where the 
posttest value of a given outcome measure was used as the dependent variable; 
independent variables included the pretest value of that measure as well as group 
assignment (Even Start or control). The results displayed in the following sections show 



2 A follow-up posttest is being conducted dur ing May-June of 1 993. Data from this second posttest 
will be analyzed for the final report fium this evaluation. 
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Exhibit 7.2 






Demographic Characteristics at Pretest of Families 


in nanaom Assignment -bites of 


the In-Depth Study 


Demographic Characteristics 


Percentage of Families 








Even Start 


Control 




(n=101) 


(n = 98) 


Family Configuration: 






Single parent 


40% 


39% 


Couple 


41% 


46% 


Extended family 


17% 


13% 


Other 


1% 


1% 


Education Level: 






Parent has high school diploma, GED, 


O ft/ 

27% 


19% 


or higher 






Spouse/partner has high school 


47% 


39% 


fiinlnma f^Pft nr Hinhor 




Emoiovment Status* 






rarent not woricing 


a o a/ 

83% 


82% 


rarant worKing part-ume 


5% 


7% 


Parent working full-time 


12% 


11% 


Spouse/partner not working 


27% 


14% 


Spouse/partner working part-time 


6% 


7% 


Spouse/partner working full-time 


67% 


79% 


rrimary oource ot income: 






Government assistance 


53% 


44% 


Job wages 


42% 


51% 


Alimony and child support 


2% 


1% 


utner 


3% 


4% 


Annual Income: 






Less than $5,000 


52% 


44% 


5,000 - 10,000 


21% 


30% 


10,000- 15,000 


13% 


S% 


15,000-20,000 


7% 


10% 


20,000 - 25,000 


3% 


6% 


More than 25,000 


4% 


1% 



Even Start 
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Exhibit 7.2 
(continued) 


Demographic Characteristics 


Percentage of Families | 


Even Start 
(n=101) 


I Control 
I (n=98) 


Primary Language of Parents: 

English 

Spanish 

Other 


63% 
36% 
0% 


54% 
45% 
1% 


Primary Language of Child: 

English 

Spanish 

Other 


68% 
32% 
0% 


59% 
40% 
1% 


Race/Ethnicity of Parent: 
White 

African-American 

Hispanic 

Other 


14% 
24% 
50% 

3% 


15% 
19% 
49% 
0% 


Age of Parent: 

Less than 20 years old 
20 - 29 years old 
30 years old or older 


3% 
68% 
29% 


2% 
76% 
22% 


Homelessness: 

Currently homeless 
Homeless within last year 


2% 

4% | 


1% 
10% 



Even Start 
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raw pretest and posttest means and standard deviations, as well as raw pre-post gains 
for Even Start and the control group. The "program effect" is the gain for the Even Start 
group, over and above the gain for the control group. The "effect size" is the program 
effect converted to standard deviation units, by dividing by the pretest standard deviation 
of the control group. 

In addition to the In-Depth Study, the NEIS provides great amounts of pretest-posttest 
data on about 120 Even Start projects, and these data are used to provide information 
about pretest-posttest gains for Even Start participants. The approach used here is 
measure-specific. For some measures we have developed our own Even Start "norms," 
for others we use external standards of comparison such as the norms that publishers 
provide for standardized tests, the scores attained by similar populations in other recently 
completed evaluations, or the scores or program participants prior to receiving program 
services. While these approaches are second-best to a randomized experiment, the 
combination of the two provides for a strong assessment of the short-term effects of 
Even Start. 

The challenge for this report lies in reconciling findings from the two different data sets. 
This is easy when the two data sets to the same conclusion, but is more difficult when 
one data set points to a positive effect while the other data set shows that Even Start 
makes no difference. Additional data will be available for the final report from this study, 
so the conclusions drawn here ought to be regarded as tentative. 

A listing of the outcome measures used in the In-Depth Study portion of the evaluation 
and in the NEIS is contained in Exhibit 7.3. 
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Exhibit 7.3 






Outcome Measures Used in In-Depth Study and NEIS 




Outcome Measure 


IDS 


NEIS 


wiiiiurcn 






PreSchool Inventory 


X 


X * 


reaooay riciure vocaouiary Test 


X 


X 


uniia s emergent Literacy Test 


X 


- 


Parents: Literacy levels 






Comprehensive Adult Student Assess. System 


x 


x 


Attainment of a GED 


x 


x 


Reading/writing activities at home 


x 


x 


Parents: Parenting and personal skills 






Personal skills 






Depression scale 


X 


— 


Mastery scale 


X 




Home learning environment 






Talking with child 


X 


X 


Play materials in home 


X 


X 


Number of children's books in home 


X 


X 


Reading to child 


X 


X 


neauing materials in nome 


X 


X 


Learning activities at home 




X 


Teaching child 


x 


X 


Family rules 


x 


— 


Activities with child 


■ x 




Parent as a Teacher 


x 


x 


Parent/child reading task 


x 




Parents' expectations and involvement 






expectations tor scnoot success 


X 


X 


Expectations for high school grad. 


X 


X 


Families 






Social support scale 


X 




Family resources 






Source of income 


X 




Income level 


X 




Adequacy of resources 


X 




Employment status 


X 


X 
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Section Eight 

Effects on Children 



In this evaluation, short-term effects of Even Start on children are assessed using the 
following measures: the PreSchool Inventory, which measures a range of school 
readiness skills; the Peabody Picture Vocabulary Test, which measures receptive 
vocabulary; and the Child's Emergent Literacy Test, which assesses the pre-literacy skills 
of young children. 



The PreSchool Inventory 

The PreSchool Inventory (PSI) was originally developed by Bettye Caldwell as a 64-item 
inventory of basic concepts important for preschool children to know before entering 
school (CTB/McGraw-Hill, 1970f. A 32-item version has been adapted (Abt Associates 
Inc., 1991) for use in large-scale evaluations. 



Description of the Measure 

The PS I is an individually-administered measure that assesses a range of school readiness 
skills, e.g., identifying shapes and colors and understanding numerical concepts. The PSI 
requires 15 minutes to administer and is appropriate for children between the ages of 
three and five. English and Spanish translations of the test are combined on a single 
form. Scoring is straightforward; each item correct counts as one point, and a total score 
is computed. The PSI does not contain any subscales. 

The 32-item version has been used in numerous large-scale evaluation studies including 
the observation study of Chapter 1 preschool programs (Seppanen et al., 1993), the 
evaluation of Project Giant Step (Layzer, Goodson and Layzer, 1990), the National Day 
Care Study (Bache, 1 980), the Head Start Planned Variation study (Walker, Bane and Sryk 
1 973), the National Home Start Evaluation (High/Scope Educational Research Foundation, 
1973, 1975), and the Child and Family Resource Program evaluation (Travers et al., 
1982). It also is being considered for use in the current evaluation of the Comprehensive 
Child Development Program (St. Pierre et al., 1990). 

The PSI was developed to be sensitive to instruction, and has shown positive effects of 
preschool programs in previous research, but it does not have national norms. The 
psychometric characteristics of the test have been investigated extensively. The reliability 
of the measure has been assessed in each of the studies cited above, with Cronbach's 



S-7 



Effects on Children 



alpha ranging from .77 to .87. Test-retest reliability ranged from .67 to .77. In the 
current evaluation, the reliability of the PSI as assessed via Cronbach's alpha is .86. 

For both the NEIS and the In-Depth Study, the PSI was administered to children between 
the ages of three and five who were participating in early childhood education. 
Administration rules for the NEIS were that the test was given at the beginning of the 
school year (or at any other time of entry to Even Start) and again in the spring (or at the 
time of exit from Even Start). Project staff were asked to administer the PSI as a pretest 
within 30 days of the start of services and not to administer the PSi as a posttest unless 
there were at least three months between pretest and posttest. Staff reported the PSI 
raw score, the test date, and other information about the testing situation. Since no 
score conversions are required, there are few opportunities for errors in reporting. About 
one in five pretest scores were lost because the child was tested at an inappropriate age 
or there was incomplete data on birthdate or test date which prevented calculation of age 
at the time of testing. Posttest scores were available for only about half of the children 
who were pretested. 

For the In-Depth Study, the PSI was administered by trained data collectors. A pretest 
was given when families entered Even Start in the fall of 1991 . Control group families 
were pretested at the same time as Even Start families. Both groups were posttested in 
the summer of 1992. 



Pretest Levels on the PSI 

Data from the NEIS were analyzed in order to describe PSI pretest scores for the Even 
Start population. Exhibit 8.1 presents the average PSI pretest scores attained by children 
entering Even Start, in total and broken down by several variables. Across 1,477 pretest 
scores from children who entered Even Start in the fali of 1 990 or fall of 1 991 , the mean 
PSI pretest score was 13.0 points, with a standard deviation of 6.8 points. The PSI 
measures school readiness skills, and so we expect scores to increase as children age. 
This indeed was the case. Pretest means are 10.1 points for three-year olds, 14.8 points 
for four-year olds, and 17.4 points for five-year olds. Pretest means are also presented 
for different types of family structure, for different levels of family income, for the 
language used in testing, and for the location of testing. Subgroup Differences are 
generally in the expected direction, e.g., children with prior preschool experience score 
higher than children with no preschool experience. 



Developing Age Norms for the PSI 

One way to measure the effect of Even Start on the PSI is to compare pretest-posttest 
changes observed for children in families assigned to Even Start with changes observed 
for children in families assigned to the control group in the In-Depth Study projects. 
Because families were randomly assigned to the two groups, this approach will yield an 
unbiased estimate of the effect of Even Start in these sites . But, the In-Depth Study 
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Exhibit 8,1 








PSI Pretest Scores 






(Raw Scores from the NEIS Data Set) 




Group 


N 


Mean 

I— 11- 


SD 


Age at pretest 










3-0 to 3-1 1 




611 


10.1 


5.8 


4-0 to 4-11 




819 


14.8 


6.7 


5-0 to 5-1 1 




47 


17.4 


7.1 


6-0 to 6-1 1 








- 


7-0 to 7-1 1 









- 


Gender 










Male 




696 


12.5 


6.9 


Female 




719 


13.5 


6.7 


Prior preschool experience 










No 




1,042 


12.2 


6.6 


Yes 




333 


14.8 


7.0 


Highest grade attained by target parent 








Grade 0-4 




29 


12.0 


8.6 


Grade 5-8 




243 


12.0 


6.9 


Grade 9-12 




847 


13.0 


6.7 


Diploma or GED 




293 


14.0 


6.7 


Language test administered in 










English 




1,202 


13.5 


6.7 


Spanish 




166 


10.1 


7.1 


Total 




1,477 


13.0 


6.8 


Exhibit reads: The average PS! pretest score for males was 12.5 points. 
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relies on data from only five sites, with a total of about 100 families in Even Start and 
100 control families. 

We would like to use data from the NEIS to augment the conclusions that can be drawn 
from the In-Depth Study. The problem is that the NEIS only collects data from Even Start 
families-no control families are measured, if the PSI had national norms, we could 
estimate the amount of growth to expect on the PSI by comparing the pretest-posttest 
growth of Even Start children with the growth of children nationally. But, no such norms 
exist for the PSI, and even if they did, they probably would not be based on a sample of 
children that adequately represents the Even Start population. 

Fortunately, the pretest data collected for this evaluation afforded us the opportunity to 
develop age norms for The PSI based on data collected on Even Start children . By 
definition, the resulting norms are directly applicable to the Even Start population. In 
brief, the methodology called for administering the PSI to three to five year olds as they 
entered Even Start and using these pretest scores to generate a growth curve which 
represents the no-treatment expectation for the Even Start population. Additional 
information on this approach is contained in St.Pierre et al., (1993) and Murray et aL, 
(1993). To the extent that children entering Even Start have had prior preschool 
experience, their pretest scores reflect learning obtained through that experience. 

The results of the norms development effort are summarized in Exhibit 8.2 which shows 
that children in the Even Start population are expected to gain an average of .42 items 
per month on the PSI, solely on the basis of normal development. Children who are 
administered the PSI in Spanish are expected to gain an average of .30 items per month. 
Gains are expressed in terms of number of items per month because children participate 
in Even Start for different lengths of time (different numbers of months). We use these 
no-treatment expectations in subsequent analyses to determine whether participation in 
Even Start produced pretest-posttest changes which are greater than what would be 
expected on the basis of normal development. 



Effects on the PSI 

Data from two evaluation sources, the random-assignment In-Depth Study evaluation and 
the national survey of all Even Start programs (the NEIS), are consistent in indicating that 
Even Start has a statistically significant, positive effect on the PSI-an effect 
commensurate in size with effects on the PSI that have been observed in other 
evaluations of high-quality preschool programs. 

Effects as Measured by the In-Depth Study. Data from the In-Depth Study show that 
children in families assigned to participate in Even Start gained an average of 4.5 points 
from pretest to posttest (about a nine-month period) compared to an average gain of 3.1 
points for children in families assigned to the control group (Exhibit 8.3). Both the Even 
Start and the control group gained a statistically significant amount from pretest to 
posttest, however the gain for the Even Start children was significant, over and above th . 
gain for the control children. The size of the program effect is .23 standard deviations, 
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Exhibit 8.2 






Developmental Growth on the PSI in the 
Absence of Even Start Instruction 
(Based on the NEIS Data Set) 




Group 1 


Number of 
Children 


Expected 
Growth in 
Items/Month 


95% 
Conf id enrp 

Band 


No prior preschool 
experience 


892 


.42 


.36-. 48 


Prior preschool experience 


292 


.41 


.30.53 


Administered in Spanish 


208 


.30 


.16-.43 


Exhibit reads: Without preschool experience, including Even Start, a child would be expected to gain .42 
raw score points per month. 


Note: Developmental growth is estimated from cross-sectional pretest scores. It is defined as the slope 
of the regression line predicting pretest scores from age in months. The 95 percent confidence band is 
obtained by adding and subtracting 1 .96 standard errors from the expected growth. 





Exhibit 8.3 






PreSchool Inventory: 
Effects from the In-Depth Study 






Even Start 
| (n=84) 


Control 
(n=75) 




Mean |j S.D. 


Mean 


S.D. 


Pretest 


11.4 5.9 


11.7 


6.1 


Posttest 


15.9 6.6 


14.8 


6.8 


Gain 


4.5* 


3.1* 




Program Effect 
(Effect Size) 


1 .4 points* 
(.23 s.d.) 




• p < .05 








Exhibit reads: Even Start children in the In-Depth Study averaged 1 1 .4 points on the PSI pretest. 
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a relatively small, but real, effect as judged by the standards of other social program 
evaluations. 

Effects as Measured by the NEIS. Data from the NEIS were analyzed differently than data 
from the In-Depth Study. The results show that, across all Even Start projects, children 
gained at the rate of .92 PSI items per month, compared with a developmental 
expectation of .42 items per month. Exhibit 8.4 illustrates how the PSI growth 
trajectories of children with and without Even Start diverge over time. The trajectory for 
children prior to participation in Even Start shows growth at a rate of .42 items per 
month-the rate expected on the basis of normal development. The trajectory for children 
after participating in Even Start shows growth at an accelerated rate of .92 items per 
month. 

The gain of .92 items per month can be expressed as the combination of .42 items per 
month due to normal development plus .50 items per month due to Even Start. We can 
therefore say that participation in Even Start more than doubles the expected rate of 
learning on the PSI. This is a substantial effect, one which is equivalent to the largest 
child-level gains observed on the PSI in other studies of preschool programs. 

Another way to interpret the data is to express the size of the effect in terms of standard 
deviation units. Dividing the effect of .50 items per month by the PSI's standard 
deviation of 6.2 shows that participation in Even Start has an effect size of .08 standard 
deviation units per month. Findings presented earlier in this report showed that, on 
average, children participate in Even Start for about six months. Thus, the NEIS data 
show that Even Start has an effect of .48 standard deviation units for the average Even 
Start child. This is a "medium-sized" effect by general standards of social science 
evaluations. 

Exhibit 8.5 provides a comparison of the effect observed on the PSI in the Even Start 
evaluation with effects on the PSI as seen in four other large-scale evaluations of early 
childhood education programs conducted in the 1970s and 1980s. In the other 
evaluations, the developmental gain (no-treatment expectation) on the PSI ranged from 
.4 to .5 items per month, and the gain including the effect of the program under study 
ranged from .6 to 1 .0 items per month. Children participating in Even Start fit the 
developmental pattern observed in the other studies exactly, gaining .42 items per month 
prior to entering Even Start. Once in Even Start, they gained at an improved rate of .92 
items per month. This accelerated rate of learning on the PSI means that as Even Start 
children enter the public schools they are more likely to know basic concepts and 
precursors of kindergarten skills than they would have been in the absence of the 
program. 

Information on how different subsets of Even Start children performed on the PSI is 
presented in Exhibit 8.6. The mean monthly gains are remarkably stable across 
subgroups. That is, childrens' monthly gains on the PSI are roughly equivalent, regardless 
of age of child, gender, ethnic background, prior preschool experience, education level of 
parent, language in which the test was administered, and location of the pretest. This 
argues that Even Start does equally well at teaching school readiness skills to quite varied 
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Exhibit 8 



.4: PSI Growth for a Child Entering Even Start at Age Three Compared 
with Developmental Growth Estimated from Pretest Scores 




3-0 



3-6 



4-0 

Age (Years-Months) 



4-6 



5-0 



Exhibit reads: A child entering Even Start at age three would be expected to gain .92 items per month on the 
PSI compared with the .42 items per month growth estimated from cross-sectional data from children without 
prior preschool experience. 



Exhibit 8.5: Size of Developmental and Program 
Gains on the PSI in Several Studies 



I0.58T 




Legend 

Q Program GUin 

E5 n«»ilnpiii«nal QHn 



Exhibit reads: Children in Even Start gained .92 points per month on the PSI. 
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Exhibit ft 6 

Average Monthly Gain in Raw Score 
Points on the PSI for Selected Variables 
(NEIS Data Set) 


Monthly Gain 


Group 


N 


Mean 


SD 


Age at pretest 








3-0 to 3-1 1 


566 


.86 


.85 


4-0 to 4-1 1 


456 


.99 


.83 


Gender 








Male 


490 


.91 


.91 


Female 


481 


.92 


.78 


Ethnic background 








Asian 


10 


1.23 


.72 


African-American 


292 


.92 


.94 


Hispanic 


224 


.90 


.89 


Native American 


30 


1.08 


.94 


White 


438 


.92 


.76 


Prior preschool experience 








No 


761 


.93 


.85 


Yes 


163 


.88 


.86 


Highest grade attained by target parent 








Grade 0-4 


22 


1.25 


.77 


Grade 5-8 


189 


.90 


.90 


Grade 9-12 


571 


.91 


.86 


Diploma or GED 


184 


.95 


.71 


Language with which pretest was administered 








English 


855 


.91 


.83 


Spanish 


126 


.90 


.96 


Location of pretest 








Center 


556 


.97 


• 84 . 


Home 


326 


.88 


.85 


Other 


1 14 


.77 


.83 


Total 


1,022 


.92 


.85 


Exhibit reads: Children tested twice with the PSI gained an average of .92 raw score points per month. Since 
developmental growth accounts for about .42 raw score points per month. Even Start early childhood education 
more than doubles the growth rate of readiness skills for participating children. 
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groups of children, and that the overall estimate of .92 items per month is a robust 
indicator. Where it appears that large differences exist between subgroups, the sample 
size in one of the subgroups tends to be small, casting doubt on the reliability of the 
estimate. 



The Peabody Picture Vocabulary Test 

The Peabody Picture Vocabulary Test-Revised (PPVT) measures receptive (hearing) 
vocabulary, and gives a quick estimate of verbal or literacy-related skills. 



Description of the Measure 

The PPVT is an individually administered test which requires 15 to 20 minutes per child 
and is appropriate for children between the ages of two and 1 8 years. In this evaluation, 
the PPVT is used with children three years of age and older. The test consists of 175 
vocabulary items of increasing difficulty. The tester reads a word and the child selects 
one of four pictures that best describes its meaning. A total score is calculated based on 
the number of words correctly identified; no subscales exist. Whereas the PS! measures 
school readiness, the PPVT assesses verbal/language skills. 

The PPVT was standardized in 1979 on a national sample based on the 1970 U.S. 
Census. The standardization sample included 4,200 individuals from a variety of 
demographic backgrounds who were between 30 months and 18 years of age (Dunn and 
Dunn, 1981). In addition, the PPVT is available and normed for Spanish-speaking 
individuals (Dunn et al., 1986). The Spanish version (the Test de Vocabulario en 
Imagenes Peabody, or TVIP) was standardized in 1981-82 on over 2,000 children in 
Mexico and Puerto Rico. The PPVT-R and PPVT have been used successfully in a number 
of large-scale research studies and surveys including the National Longitudinal Study of 
Youth (Baker and Mott, 1989), the Infant Health and Development Program (IHDP, 1990), 
and the evaluation of the Comprehensive Child Development Program (St.Pierre et al., 
1990). 

The PPVT-R manual reports split-half reliabilities of .80 for Form Land .81 for Form M. 
TVIP reliabilities are higher (.93). The immediate test-retest reliability is .82, while the 
delayed test-retest reliability is .78. A great deal of validity information is available for the 
PPVT-R and the PPVT. Issues of content validity, construct validity, and criterion-related 
validity all are addressed in the PPVT-R manual. Conclusions to be drawn from these 
analyses are that the PPVT correlates most highly with other measures of vocabulary 
(average correlation of .71 ), it correlates moderately well with tests of scholastic aptitude 
(about .5 - .6), and it correlates reasonably well with measures of school achievement (.3 
- .7) but does less well as a predictive measure of school success. 

For this evaluation, PPVT data are expressed in standard score units, rather than raw 
scores or percentile ranks. Standard scores express, in standard deviation units, the 
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extent to which a child's score exceeds, or falls below, the mean score of children of the 
same age upon whom the test was standardized. PPVT standard scores have a mean of 
100 and standard deviation of 15. So, for example, three-year-old children in the PPVT 
norms group have an average raw score of 25, which equates to a standard score of 100; 
four-year-olds have an average score of 42, which also is assigned a standard score of 
100; five-year-old children have an average score of 57, which equates to a standard 
score of 100; and so on. 

For the NEIS, project staff were asked to administer Form L of the PPVT-R within 30 days 
of entry to Even Start and again in the spring (or when the family exited the program). 
Alternate forms were to be used at each testing (Form L followed by Form M). Children 
were tested with the TVIP if, in the judgement of the program staff, this was a more 
appropriate test. Screening of completed tests revealed a high rate of error on the part 
of project staff in converting raw scores to standard scores, so all conversions were 
verified. Project staff appear to have had difficulty using the conversion tables accurately. 

For the In-Depth Study, PPVT administration was done by trained data collectors 
following the same rules that were described for the PSi. 



Norms for the PPVT and TVIP 

The PPVT and TVIP each have national norms. The norms are used to translate raw 
scores to standard scores, which can then be translated into percentile ranks. Analyses 
presented in this report are done on standard scores, and because of the way that 
standard scores are constructed, our expectation is that PPVT standard scores should not 
change in the absence of a "treatment." There is no "maturation effect" for the PPVT as 
there was for the PSI, because the standard scores are age-linked. That is, a three-year- 
old who scores at the mean for all three-year-olds will have the same standard score as 
a five year old who scores at the mean for all five-year-olds. Hence, there is no particular 
reason that a child's percentile ranking relative to the PPVT (or TVIP) norms group should 
change over time unless that child is receiving some special services. An increase in 
standard scores during the time that a child is participating in Even Start is, therefore, 
taken as an indication that Even Start is helping to increase children's receptive 
vocabulary. 

To test our hypothesis that there is no relationship between age and PPVT standard 
scores we performed an analysis of pretext data from the NEIS, similar to the analysis 
that was done to derive age norms for the HSI. Exhibit 8.7 is a graph of age at pretest 
by PPVT standard scores. As expected, pretest standard scores are quite stable across 
ages. Four year olds seem to score a bit lower than expected, and six year olds score 
somewhat higher, but the differences are not large. This means that we do not expect 
to see increases or decreases in PPVT standard scores due to maturation or development. 
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Exhibit 8.7: Average PPVT Pretest 
Standard Scores by Age in Months 
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Effects on the PPVT 

Unlike the PSI, results from the In-Depth Study and the NEIS tell different stories for the 
PPVT. Findings from the In-Depth Study show no significant program effect. On the 
other hand, results from the NEIS indicate that there may be a positive effect of 
participating in Even Start. The fact that the In-Depth Study results do not support the 
NEIS findings makes us wary about reaching firm conclusions at this time about the effect 
of Even Start on the PPVT. Additional data which may lead to firmer conclusions will be 
presented in the final report from this evaluation. 



Effects as Measured by the In-Depth Study. The small pretest-posttest gains on the PPVT 
for Even Start and for the control group are not statistically significant, nor is the 
estimated program effect (Exhibit 8.8). 

Effects as Measured by the NEIS. Data from the NEIS lead to a different conclusion than 
that which was drawn from the In-Depth Study data. Based on the NEIS, we see that 
Even Start children make significant gains on the PPVT. Exhibits 8.9 and 8.10 present 
the results of analyses based on NEIS data from all Even Start projects. The exhibits 
show average pretest-posttest gains on the PPVT and TVIP, expressed in standard score 
points per month (gains are expressed in points per month because children participate 
for different numbers of months). 
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Exhibit 8.8 








Peabody Picture Vocabulary Test: 
Effects from the In-Depth Study 








Even Start 
(n = 84) 


Control 
(n = 75) 




Mean 


S.D. 


Mean 


S.D. 


Pretest 


77.8 


19.5 


77.0 


17.1 


Posttest 


80.7 


16.5 


77.9 


15.8 


Gain 


2.9 




0.9 




Program Effect 
{Effect Size) 




2.0 points 
{.12 s.d.) 




* P < .05 










Exhibit reads: Even Start children in the In-Depth Study averaged 77.8 points on the PPVT pretest 



Based on the norms analyses described above, we do not expect to see any change in 
PPVT standard scores due to maturation. However, we do see that children participating 
in Even Start gain an average of .83 standard score points a month on the PPVT and 1 .28 
points per month on the TVIP. Dividing by the PPVT standard deviation yields a per 
month effect of .05 standard deviation units for the PPVT and .08 standard deviation 
units for the TVIP. Multiplying by six months, the average length of participation in Even 
Start, yields an effect size of .30 standard deviations for the PPVT and .48 standard 
deviation units for the TVIP-medium-sized effects compared to the effects of other social 
science programs. 

Exhibit 8.9 shows monthly gains for several different subgroups of children. For example, 
children who are four or five years of age gain more than children who are three years old! 
This fits with data presented earlier in this report which showed that four-year-olds 
received substantially more early childhood education instruction than three-year-olds. 
However, the fact that there are large variations in monthly gains across subgroups of 
children for the PPVT suggests that Even Start may be differentially effective at improving 
receptive vocabulary for some groups of children. This was not the case on the PSI, 
where Even Start seemed to affect school readiness equally for all subgroups of children! 

One possible explanation for the contrary PPVT findings across the NEIS and the In-Depth 
Study has to do with the different racial/ethnic distributions in these two parts of the 
evaluation. The NEIS data set is based on ail Even Start projects, in which 42 percent of 
the children are white, 30 percent are African-American, and 19 percent are Hispanic. 
On the other hand, the random-assignment In-Depth Study is conducted in five projects 
which overrepresent Hispanics when compared to the total Even Start population. In the 
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Exhibit 8.9 
Average Monthly Gain on the PPVT 

iii wuaiiuoi u ocorc ruinis TOr o6ieCT60 VdnaDI8$ 

(NEIS Data Set) 


Monthly Gain 


uroup 


« l 


Mean II 

ii 


SO 


Age at pretest 








3-0 to 3-1 1 


460 


.31 


1 8? 


4-0 to 4-1 1 


720 


1 .26 


? 1 "3 

1 o 


5-0 to 5-1 1 


307 


.81 


1 91 


6-0 to 6-1 1 


147 


.46 


2 Oft 


7-0 to 7-1 1 


42 


.58 


1 


Gender 








Male 


872 


.81 


2 05 


Female 


795 


.78 


2 05 


Ethnic background 








Asian 


26 


1.09 


2 09 

dm ,\J k) 


African-American 


556 


1.11 


o ia 


Hispanic 


209 


.59 


2.17 


Native American 


123 


.27 


1 62 


White 


757 


.72 


1.90 


Prior preschool experience 








No 


543 


.99 


2 14. 

dm, 1 *T 


Yes 


543 


.88 


2.12 


Highest grade attained by target parent 








Grade 0-4 


16 


65 




Grade 5-8 


253 


.81 


2.28 


Grade 9-1 2 


995 


.82 


2.04 


Diploma or GED 


374 


.84 


1.90 


Form administered at pretest 








Form L 


1,638 


.85 


2.04 


Form M 


82 


.07 


2.02 


TOTAL 


1,676 


.83 


2.04 


Exhibit reads: Children tested twice with the PPVT had a pretest average of 82 (equivalent to a percentile rank of 
12) and an average monthly gain of .83 in standard score points. This monthly gain should reflect only the added 
effect of Even Start since the age norms adiust for developmental growth by setting the average standard score to 
1 00 for each age group. 
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Average Monthly Gain on the TVIP 
in Standard Score Points by Age at Pretest 




Age at Pretest 


Monthly Gain 


N 


Mean 


SD 


3-0 to 3-1 1 


33 


.99 


2.60 


4-0 to 4-1 1 


71 


1.39 


2.38 


5-0 to 5-1 1 


38 


1.58 


2.16 


6-0 to 6-1 1 


25 


1.05 


2.69 


7-0 to 7-11 


5 


.53 


2.36 


Total 


172 


1.28 


2.41 


Exhibit reads: Children tested twice with the TVIP had a pretest average of 93 (equivalent to a 
percentile rank of 32) and an average monthly gain of 1 .28 in standard score points. 


Note: Combined norms {Mexico and Puerto Rico) were used in obtaining standard scores. 





In-Depth Study, 14 percent of the children are white, 24 percent are African-American, 
and 50 percent are Hispanic. 

The differences in racial/ethnic background are important because monthly PPVT gains 
differ by racial/ethnic group: 1.11 standard score points per month for African-American 
children, .72 points per month for white children, and .59 points per month for Hispanic 
children (Exhibit 8.10). Because the In-Depth Study contains a larger percentage of 
Hispanic children than the NEIS, we would expect the effect of Even Start on the PPVT 
to be smaller in the In-Depth Study than in the overall Even Start population. 
Unfortunately, the In-Depth Study has a relatively small sample size, making it difficult to 
detect small effects. 

Therefore, it is possible that Even Start is having a positive effect on the PPVT, that we 
are seeing that effect on the NEIS, but that we are not able to detect the effect in the In- 
Depth Study because of a combination of small sample size and overrepresentation of 
Hispanic children who, on average, do not gain as much on the PPVT as African-American 
or white children. 



Child's Emergent Literacy Test 

In addition to the PSI and PPVT, we assessed children ~ the In-Depth Study using a brief 
set of items designed to tap children's emergent literacy skills. This measure was not 
used in the NEIS. 
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Description of the Measure 



Children in literate homes begin to learn to read and write very early in life. At two and 
three years of age they encounter print and have ideas about the process and purposes 
of reading and writing. Young children are involved in their own literacy development 
through social interaction with literate parents or other literate persons, through 
explorations into written language, and through observing the literacy practices of adult 
models. Emergent literacy constructs include the child's knowledge of some basic 
information (ordinal position, colors, shapes) as well as more specific information about 
book and print knowledge. 

No existing emergent literacy measure reviewed was ideally suited to Even Start 
participants (Teale 1986a, 1986b). Therefore, a simple assessment of emergent literacy 
was created from existing instruments«the Child's Emergent Literacy Test (Abt 
Associates, 1991). It includes items assessing orientation and directionality of text, 
recognition of letters and punctuation, purposes of reading, and children's knowledge and 
skills in writing their own name and age. The CELT was administered to children by 
trained data collectors in an informal face-to-face setting. It requires only a couple of 
minutes to administer. The testing was done at the same timep Mnts as the PSI and 
PPVT. 

Because the CELT was created for this evaluation, there is no history of psychometric 
characteristics. The reliability (Cronbach's alpha) of the CELT as administered in this 
evaluation is .76. The CELT consists of 1 6 items, each is scored correct or incorrect, and 
a total score ranging from zero to 16 is obtained. 



Pretest Levels on the CELT 

Children in Even Start and in the control group scored quite similarly on the CELT at 
pretest: the Even Start pretest mean was 4.3 and the control group pretest mean was 
4.5 (see Exhibit 8.1 1 ). Standard deviations were almost identical; 2.8 for Even Start and 
3.0 for the control group. We have no basis of comparison for these pretest scores since 
the instrument has not been used in other studies. We should note that ceiling effects 
are not a problem, since pretest means are between four and five points out of a total of 
1 6. However, the low pretest scores may indicate that the measure is too difficult, and 
hence is not sensitive to gains for children at this age. 



Effects on the CELT 

Children in the Even Start group gained one point on the CELT from pretest to posttest~a 
statistically significant amount. However, children in the control group also gained a small 
amount, and the program effect (the gain of Even Start children over and above the gain 
of control group children) was not statistically significant. The In-Depth Study design 
calls for retesting all children with a second posttest, and it is possible that the follow-up 
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Exhibit 8.11 






Child's Emergent Literacy Test: 
Effects from the In-Depth Study 




! 


Even Start 
(n = 84) 


Control 
(n=75) 




Mean 


S.D. 


i| 

Mean S.D. 


Pretest 


4.3 


2.8 


4.5 3.0 


Posttest 


5.3 


2.9 


5.1 3.0 


Gain 


1.0* 




0.6 


Program Effect 
(Effect Size) 




0.4 points 
(.13s.d.) 


# p < .05 








Exhibit reads: Even Start children in the In-Depth Study averaged 4.3 points on the CELT pretest 



testing will lead to somewhat different findings. The final evaluation report will address 
this issue. 



Conclusions About Effects on Children 

This evaluation assessed effects on children using three different measures: (1) the Pre- 
school Inventory, a measure of school readiness; (2) the Peabody Picture Vocabulary 
Test, a measure of hearing vocabulary; and (3) the Child's Emergent Literacy Test, a 
measure of emergent literacy skills. 

Even Start has a clear, positive effect on the PSI. Data from the random-assignment In- 
Depth Study and the NEIS survey are consistent in indicating that Even Start has a 
statistically significant, positive effect on the PSI~an effect commensurate in size with 
effects on the PSI that have been observed in other evaluations of high quality preschool 
programs. Further, effects on the PSI are quite consistent regardless of age of child, 
gender, ethnic background, prior preschool experience, education level of parent, location 
of the pretest, or language in which the test was administered. Thus, Even Start does 
quite well at teaching school readiness skills to varied groups of children. 

Findings on the PPVT are more mixed. Results from the In-Depth Study show no 
significant program effect. On the other hand, data from the NEIS indicate that Even 
Start children gain more than would be expected without the program. It is possible that 
Even Start is having a positive effect on the PPVT, that we are seeing that effect on the 
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NEIS, but that we are not able to detect the effect in the In-Depth Study because of a 
combination of small sample size and overre presentation of Hispanic children who, on 
average, do not gain as much on the PPVT as black children or white children. Additional 
data which may lead to firmer conclusions will be presented in the final report from this 
evaluation. Data from the NEIS also show us tnat the kinds of outcomes measured by 
the PPVT (vocabulary development) vary among subgroups of children. Additional 
analyses will be undertaken to determine whether variation in PPVT scores can be 
attributed to program characteristics. 

Finally, Even Start children in the In-Depth Study gained a significant amount on the 
Child's Emergent Literacy Test. However, children in the control group also gained a 
small amount, and there is no significant program effect. As for the PPVT, the results 
based on follow-up testing will be presented in the final report from this study. 
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Section Nine 

Effects on Parent Literacy 



This evaluation measured the effects of Even Start on the literacy skills of parents in the 
following areas: (1 ) functional literacy level on a reading test, (2) the percentage of Even 
Start adults who obtained their GED certificate, and (3) parental reports of their own 
reading and writing activities in the home. 



The Comprehensive Adult Student Assessment System 
(CASAS) 

The Comprehensive Adult Student Assessment System (CASAS) is an adult-oriented 
functional assessment system that measures a broad range of adult literacy skills and 
their application in real life domains including consumer economics, government and law, 
occupational knowledge, community resources, and health (Rickard et al, 1990)! 
Although the CASAS measures reading, writing, math, and problem solving skills, this 
evaluation used only the Reading Survey achievement test in order to reduce respondent 
burden and because we expected Even Start's effects to be more prominent in reading 
than in math. 



Description of the Measure 

The CASAS has the flexibility to measure participants involved in diverse adult education 
programs, spanning the range from non-readers to adults at the GED or high school level. 
An untimed paper-and-pencil test, the CASAS Reading Survey may take as long as 60 
minutes to complete. 

The CASAS has been used with adult education learners in 27 states. The test is used 
in adult education and in job training programs, with both native and non-native English 
speakers. It also has been accepted as a project in the National Diffusion Network. The 
CASAS is being used in the National Evaluation of Adult Education Programs 
(Development Associates, 1 992), in the evaluation of California's GAIN program (CASAS, 

1990) , and in the evaluation of California's 321 adult education programs (CASAS 

1991) . 

CASAS reading scores range from 1 50 to 260. The test developers suggest the following 
interpretation of CASAS scale scores: 
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Beginning literacy (below 200): Adults scoring below a scale score of 200 have 
difficulty with the basic literacy skills needed to function in an employment setting 
and in the community. While these adults can handle routine, entry-level jobs, 
they may have trouble following simple directions and safety procedures, 

Basic literacy (200 through 214): Adults scoring between scale scores 200 and 
214 can function in entry-level jobs that require only minimal literacy skills. They 
can fill out simple applications. 

Intermediate literacy (21 5 through 224): Aduits scoring between scale scores 215 
and 224 are able to perform basic literacy tasks in a functional employment 
setting. They are generally able to function in jobs or job training that involves 
following written instructions and diagrams, through they usually have trouble 
following complex sets of directions. 

High school literacy (225 and above): Adults scoring above a scale score of 224 
can usually perform work that involves written directions in familiar and some 
unfamiliar situations. They can function at a high school entry level in basic 
reading and, if they do not have a high school diploma, can profit from instruction 
in General Education Development and have a high probability of passing the GED 
test in a short time. 

Sticht (1990) found these interpretations to be reasonable and reported general 
correspondence between CASAS scale scores above 225 and the ninth to twelfth grade 
reading levels on the Tests of Adult Basic Education and the Adult Basic Learning 
Examination. We have only found spotty data on the psychometric characteristics of the 
CASAS. A correlation of .70 between the CASAS reading test and the ABLE was 
reported in unpublished data. To obtain an estimate of test-retest reliability we calculated 
the correlation between pretest and posttest scores for adults who were posttested less 
than 90 days after the pretest. The correlation was .86, suggesting that the CASAS is 
a reliable measure. The true test-retest reliability might be even higher since this estimate 
is based on data from both forms of the CASAS. 

For the NEIS, Even Start staff administered the CASAS Reading Survey which has four 
levels, A through D, with two forms of 24 to 40 items per level. Scale scores link the 
levels into a continuous scale of achievement. Staff administered a short "locator" test 
to assist in identifying the appropriate level of the CASAS. There is no Spanish version 
of the CASAS, and project staff were instructed to use their own judgment as to the 
appropriateness of administering the test to adults with limited reading skills in English. 
For the In-Depth Study, the CASAS was administered by trained data collectors using the 
same schedule and rules as all other In-Depth Study measures. 



Effects on the CASAS 

Data to assess program effects on the CASAS are available from the In-Depth Study and 
from the NEIS. The NEIS data show pre-post gains that are statistically significant but 
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relatively small, and that are about the same size as those seen in other large-scale 
evaluations of adult education programs. The In-Depth Study data does not show 
statistically significant gains on the CASAS. Both data sets show a significant 
relationship between amount of adult education instruction and gains on the CASAS. 



Effects as Measured by the In-Depth Study. Adults in Even Start gained 1 .9 points on 
the CASAS, compared to 1 .2 points for adults in the control group (Exhibit 9.1 ). Neither 
of these average gains was statistically significant, nor was the difference in the gains 
significant. Some control group members participated in adult education programs, 
however the findings did not change when the data were reanalyzed excluding those 
adults. 





Exhibit 9.1 






CASAS Reading Survey: 
Effects from the In-Depth Study 
I (Scale Score Points) 






Even Start 
(n = 84) 


Control 
(n=75) 




Mean || S.D. 


Mean 


S.D. 


Pretest 


227.4 14.3 


223.7 


19.2 


Posttest 


229.3 13.0 


224.9 


16.7 


Gain 


1.9 


1.2 




Program Effect 
(Effect Size) 


0.7 points 
(.05 s.d.) 




• p < .05 








Exhibit reads: Even Start adults in the In-Depth Study averaged 227.4 points on the CASAS 
Reading Survey pretest. 



Effects as Measured by the NEIS. Analyses of data from the NEIS were limited to all 
adults who had a valid pretest and posttest, who had at least three months between 
pretest and posttest, and who participated in at least 70 hours of adult education 
instruction between pretest and posttest. These criteria ensured that there was adequate 
time between test points for gains to occur, and that there was sufficient instruction to 
produce gains (the CASAS developers do not expect to see educationally significant gains 
with less than 70 to 100 hours of instruction). 
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If Even Start has had a positive effect on adult literacy, there should be improvement on 
the CASAS scale scores between pretest and posttest. We do not have a control group 
for the NEIS data, nor do we have a theory (as we do for the PSI or PPVT) that a portion 
of observed pre-post changes are due to some developmental process. Rather, we have 
to assume that any observed gain is probably due to Even Start since it is unlikeiy that 
adult literacy would improve without participation in a formal adult education program. 

Analysis of data from the NEIS shows that shows that adults participating in Even Start 
for 70 hours or more over at least a three month period gained an average of 3.6 scale 
score points on the CASAS. This gain was statistically significant, not a surprising result 
^iven the large number of adults included in the analysis. The effect size for the CASAS 
(gain divided by the CASAS standard deviation) is .26 standard deviation units, a value 
that is generally considered small in educational research (Exhibit 9.2). 



Exhibit 9.2 








CASAS Reading Survey: 
Gains from the NEIS 
(Scale Score Points) 






Group 


Gain 


N 


Mean 


S.D. 


Gender 








Male 


46 


5.1 


10.5 


Female 


748 


3.5 


8.5 


Ethnic Background 








Asian 


11 


3.3 


16.0 


African-American 


274 


3.1 


8.9 


Hispanic 


159 


5.6 


8.7 


Native American 


45 


3.2 


9.0 


White 


315 


3.3 


8.2 


TOTAL 


325 


3.6 


8.6 


Exhibit reads: Adults participating in 70 or more hours of adult education gained 3.6 scale score 
points on the CASAS reading. Differences between groups based on gender and ethnic 
background were not significant after adjusting for pretest differences. 
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CASAS Geins in Even Start vs. Other Adult Education Programs. Given this relatively 
small gain, and the fact that the In-Depth Study does not show any program effect, we 
ask whether this finding is unique to Even Start or if other adult education programs also 
have difficulty producing measurable gains. There have been few large-scale evaluations 
of adult education programs which contain outcome data that may be used for 
comparison purposes. Exhibit 9.3 compares the effect size for the CASAS gains of Even 
Start adults with the results from studies of other programs. 

In one evaluation of federal 321 -funded adult education programs in California, 
researchers (CASAS, 1992) found average gains of 3.0 scale score points (effect size = 
.20) for 973 students participating in adult basic education and 3.3 points (effect size 
= .22) for 1,757 students participating in English as a second language programs. All 
students took the CASAS Reading Survey after 80 to 1 00 hours of instruction. An earlier 
study of state-funded compensatory-based adult education programs in California (CBAE, 
1987) found gains of 4.2 scale score points (effect size = .29) after 100 hours of 
instruction, though the measure was the CASAS Survey Achievement battery which 
includes fewer items to measure a wider range of literacy skills than the Reading Survey 
test. In an evaluation of the Kenan model which is used by many Even Start projects, 
researchers (Darling and Hayes, 1989) found small gains on the reading subtest of the 
TABE (effect size = .29) over one project year covering an unspecified number of 
instructional hours. 

These results show that Even Start is of comparable effectiveness to other adult 
education programs in improving the literacy skills of participating adults. This is 
reasonable since many Even Start projects use services provided by local adult education 
programs for this core service. 



CASAS Gains by Hours of Instruction. The wide range in the length of time that families 
participate in Even Start and in the intensity of the adult education instruction provided 
by projects furnishes us an opportunity to strengthen our conclusions about CASAS gains. 
If adults with many hours of adult education instruction improve more than adults who 
participate only for a few hours, we can be more confident that the instruction (and 
hence. Even Start) is responsible for this growth. It also is important to examine the 
effect of entry level on CASAS gains. Due to "ceiling effects" we expect smaller gains 
for adults starting near the highest valid CASAS score than for adults entering with lower 
entry scores. 

Exhibit 9.4 shows CASAS reading gains for subgroups of adults defined by the number 
of hours of adult education. Data are shown both for the NEIS and the in-Depth Study. 
Looking at data from the NEIS, adults who participated for less than 70 hours gained 
about 2.5 CASAS scale score points, adults who participated for 70-200 hours had an 
average gain of 3.2 points, and adults who participated for over 200 hours gained an 
average of 4.9 points. Data from the In-Depth Study show somewhat smaller gains, but 
the relationship between amount of instruction and CASAS scores is completely 
consistent with data from the NEIS. Further, this relationship is statistically significant 
in both data sets and provides important evidence that the adult education instruction 
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Exhibit 9,3: Comparison of Effect Sizes for Even Start (NEIS Data Set) 
and Three Adult Education Programs 




Exhibit reads: The effect size observed on the CASAS suggests that Even Start is about as effective in 
improving adult literacy as other adult education programs. 



provided through Even Start may well be responsible for the gains reported earlier. 

Taken together, the pretest and hours of adult education instruction accounted for about 
86 percent of the variation in CASAS posttest scores. Exhibit 9.5 illustrates the 
relationship by estimating CASAS gains for selected entry level scale scores and hours 
of instruction. As can be seen, the greatest gains can be made with adults who enter 
with low CASAS scores. In fact, the exhibit shows that changes in posttest scores are 
much more sensitive to the starting point of the adult than they are to the number of 
hours of adult education instruction. The implication of this finding is that programs such 
as Even Start will have the greatest overall impact by recruiting very low-literate adults 
and providing them with a modest amount of instruction (about 50 hours). Providing 
additional instruction to these same adults results in diminishing returns, as measured by 
gains on tho CASAS. 
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CASAS Reading Gains by Hours of Instruction 
(Scale Score Points) 




Hours of Adult 
Education 
Instruction 


Number of 
Adults 


Average 
Pretest 


Average 
Gain 


Effect Size 


NEIS Data 










1-10 hours 


125 


228.9 


2.5 


1 ft 


11-69 hours 


460 


228.8 


2.5 


.17 


70-200 hours 


293 


229.5 


3.2 


.22 


Over 200 
hours 


113 


230.6 


4.9 


.34 


In-Depth Study 
Data 










1-69 hours 


74 


226.3 


1.0 


.07 


70-200 hours 


50 


231.9 


2.5 


.17 


Over 200 
hours 


27 


231.9 


3.1 


.22 


Note: In-Depth Study data for this exhibit is based on adutts in both the five random-assignment 
sites as well as five sites where random assignment was not followed. 


Exhibit reads: Higher gains on the CASAS were observed for adults with more than 70 hours of 
adult education. Even higher gains were observed with more than 200 hours. 



Attainment of A General Education Development 
Certificate 



Even Start projects promote attainment of a high school diploma or a high school 
equivalency diploma such as the GED, as do most adult education programs. The 
rationale for this goal is based upon a belief that a diploma or GED increases the chance 
the student will find employment, provides opportunities for higher wages, and improves 
the quality of life through enhanced self-esteem. 



Description of the Measure 

For the NEIS, projects reported each June whether Even Start adults attained a diploma 
or GED during the reporting year. We calculated the percentage of Even Start adults who 
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Exhibit 9.5 



Expected CASAS Reading Gains 
by Entry Level and Hours of Instruction 
(Scale Score Points; NEIS Data Set) 



Entry CASAS Scale 
Score 



Hours of Adult Education Instruction 



50 



100 



150 



200 



250 



300 



200 


8.9 


9.2 


9.6 


9.9 


10.3 


10.6 


210 


6.7 


7.1 


7.4 


7.8 


8.1 


8.5 


220 


4.6 


4.9 


5.3 


5.6 


6.0 


6.3 


230 


2.4 


2.8 


3.1 


3.5 


3.8 


4.2 


240 


0.3 


0.6 


1.0 


1.3 


1.7 


2.0 



Note: These expected gains are in scale score points. The estimates are based upon a regression 
equation (R = .926) of: posttest - ( 51.546 + .785 x pretest + .0069 x hours ) 

Exhibit reads: Adults who entered Even Start with a CASAS score of 200 would be expected to gain 
9.2 points after 100 hours of adult education instruction. 



attamed a diploma or GED during the year. Since adults attending a secondary school do 
not qualify for Even Start, we can assume that the reported data generally reflected 
attainment of a GED, although other alternative diplomas are possible in some states. For 
the In-Depth Study, we calculated the percentage of adults in Even Start and in the 
control group who received their GED in the time between entry to the study and the time 
a posttest was administered, nine months later. For both the NEIS and the In-Depth 
Study, we know that some adults participating in the evaluation had already received a 
high school diploma or a GED at the time of the pretest. The.*' adults were excluded 
from the analyses described here. 



Effects on GED Attainment 

Findings from both the In-Depth Study and the NEIS show that Even Start has had a 
statistically significant, positive effect on the percentage of adults who entered Even Start 
without a GED or a high school diploma and attain a GED. 



Effects as Measured by the In-Depth Study. Data from the In-Depth Study show that 
14.3 percent of participating adults in Even Start families attained a GED c< spared to 3.6 
percent of participating adults in control group families (Exhibit 9.6). This is a statistically 
significant effect which appears to be educationally meaningful. 
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Exhibit 9.6: Percentage of Adults Attaining a GED 
or Diploma: In-Depth Study and NEIS 




Exhibit reads: Across the 1 20 projects participating in the evaluation, 7.1 percent of the adutts who participated 
in Even Start attained a GED or diploma. 

Effects as Measured by the NEIS. Data from the NEIS show that, across the 1 20 Even 
Start projects participating in the evaluation, 7.1 percent of adults who entered Even Start 
without a GED or diploma achieved one while participating for a year or less in adult 
education services (Exhibit 9.6). However, attaining a GED is not a reasonable short-term 
goal for some adults in Even Start. For example, it is unlikely that an adult who enters 
Even Start with a sixth grade education will be able to achieve a GED in the relatively 
short time frrme of this study. Therefore, we conducted another analysis in which we 
restricted the sample of adults to those who entered with at least a ninth grade education 
and who participated in Even Start for at least three months. By eliminating adults who 
enter with very low education levels or who receive very small amounts of adult 
education, this restricted sample contains adults who ought to have a reasonable chance 
of attaining a GED while in Even Start. As expected, the percentage of adults attaining 
a GED is higher in the restricted sample than in the overall sample (1 1 .4 percent vs. 7.1 
percent). 



Variation in GED Attainment by Family Characteristics, Additional information about 
attainment of a GED in the NEIS data set is presented in Exhibit 9.7. There are no 
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surprises in the exhibit, which shows that several variables are related to attaining a GED. 
As would be expected, attaining a GED during the year is related to grade at intake to 
Even Start: almost no adults who entered with less than a fifth grade education attained 
a GED, compared to 8.7 percent of the adults who entered with some high school 
education. Attaining a GED also is strongly related to the adult's functional level on the 
CASAS: adults who scored at the "Beginning/ "Basic," or -Intermediate" levels on the 
CASAS pretest rarely attained a GED {.8 percent), while 12.8 percent of adults who 
scored at the "High School" level on the CASAS pretest attained a GED during the year. 
Total hours of adult education instruction is another variable that is related to attaining 
a GED: adults who had higher amounts of instruction were more likely to attain a GED 
than adults with lower amounts of instruction. 



Variation in GED Attainment Across Projects. Given the great variation among projects 
in the characteristics of adults served and the amount of adult education instruction 
received by participating adults, we expect to see substantial variation among projects in 
GED attainment. Exhibit 9.8 shows a distribution of the percent of adults attaining a GED 
for projects in the evaluation. Only adults entering Even Start with some high school 
education and at least three months of adult education instruction were included. 
Twenty-three projects had fewer than ten adults meeting these criteria, and hence were 
excluded from the analysis. Among the remaining 100 projects, the within-project 
percentage of adults attaining a GED ranged from zero to 60 percent. Over half of the 
projects (57) had less than ten percent of their adults attaining a GED while three projects 
reported that more than 50 percent attained a GED. 



Reading and Writing Activities in the Home 

Two sets of questions were included in the In-Depth Study parent interview to ask 
parents about their own reading and writing habits in the home (these questions were not 
included in the NEIS portion of the evaluation). Both lists were adapted from a 
questionnaire used to evaluate the California State Library's Aduit Learner Program. 



Description of the Measure 

The 13 self-report questions about reading activities ask how often the parent reads 
different types of literacy materials commonly found in and around the home including 
junk mail, letters and bills, coupons, labels on food, cooking recipes, religious materials, 
instructions, street signs, newspapers, notes from school, T.V. Guide, magazines, and 
books. The 1 1 self-report questions about writing activities ask how often the parent 
writes at home in the following areas: checks, notes, recipes, forms or applications, 
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Exhibit 9.7 

Number and Percent of Participating Adults 
Who Attained a GED While Served by Even Start (NEIS Data Set) 


Group 


All adults lacking a 
diploma at intake 


Adults entering with at 
least a ninth grade 
education and participating 
more than three months 


Total 
Adults 


Attained GED 


Total 
Adults 


Attained GED 


N 


% 


N 


% 


Highest grade at intake 














Grade 0-4 


490 


2 


.4 


- 


- 


- 


Grade 5-8 


1,765 


69 


3.9 


- 


- 


- 


Grade 9-12 


5,452 


476 


8.7 


3,238 


369 


11.4 


Functional level on CASAS at pretest 










Beginning to Intermed 


1,400 


11 


.8 


599 


8 


1.3 


High School 


2,790 


352 


12.8 


1,594 


257 


16.1 


Total hours of adult education instruction 










1-10 hours 


1,230 


33 


2.6 


144 


11 


7.6 


1 1-69 hours 


3,178 


208 


6.1 


1,461 


122 


8.4 


70-200 hours 


1,619 


194 


10.7 


1,113 


151 


13.6 


> 200 hours 


658 


97 


12.8 


520 


85 


16.3 


Age at intake 














16-21 


968 


80 


8.3 


409 


52 


12.7 


21-25 


2,327 


207 


8.9 


1,048 


142 


13.5 


26-30 


2,026 


138 


6.8 


920 


100 


10.9 


31-35 


1,084 


58 


5.5 


440 


39 


8.9 


36-40 


528 


26 


5.0 


179 


17 


9.5 


Over 40 


413 


15 


3.6 


128 


10 


7.8 
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(continued) 








Numb 
Who Attained a 


er and Percent of Participating Adults 

GED While Served by Even Start (NEIS Data Set) 






All adults lacking a 
diploma at intake 


Adults entering with at 
least a ninth grade 
education and participating 
more than three months 


Group 


Total 
Adults 


Attained GED 


Total 
Adults 


Attained GED 


N 


% 


N 


% 


$15,000-20,000 


497 


53 


10.7 


214 


35 


16.4 


$20,000-25,000 


255 


30 


11.8 


120 


20 


16.7 


Over $25,000 


208 


26 


12.5 


87 


15 


17.2 


Primary language is English 












No 


2,005 


33 


1.6 


472 


20 


4.2 


Yes 


5,568 


512 


9.2 


2,702 


347 


12.8 


Ethnic background 














Asian 


148 


2 


1.4 


42 


2 


4.8 


African-American 


2,287 


86 


3.8 


1,105 


57 


5.2 


Hispanic 


1,885 


46 


2.4 


462 


32 


6.9 


Native American 


468 


42 


9.0 


227 


22 


9.7 


White 


2,746 


368 


13.4 


1,340 


253 


18.9 


TOTAL 


7.707 


547 


7.1 


3,238 


369 


11.4 


Exhibit reads: Among the 7,707 adults lacking a high school diploma or GED at intake, 7.1 percent 
attained a GED while participating in Even Start. 
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Exhibit 9.8: Project-Level Distribution of the 
Percentage of Adults Who Attained a GED (NEIS Data Set) 




Percent of Adults Attaining GED 



Exhibit reads: Less than five percent of the adults attained a GED in 42 projects while 50 to 55 percent attained 
a GED in three projects. Only adults with nine to 12 years of schooling at intake and at least three months 
participation in adult education were included in the analysis, 

dates on a calendar, letters, stories or poems, greeting cards, crossword puzzles, grocery 
lists, and a journal or diary. 

These questions are intended to give an indication of the literacy environment in the 
home. Since most Even Start parents are expected to be poor readers, it is not realistic 
to expect them to be avid readers and writers. However, it is not known how much they 
use reading and writing in common household activities. If few of these activities are 
undertaken, then these homes would truly be impoverished literacy environments. On the 
other hand, an increase in the use of these simple literacy tasks would be a realistic goal 
for adults who are working to improve poor reading skills. 

Parents responded to each of the questions on a three-point rating scale where a value 
of 1 meant that the activity was done "not at all," a value of 2 meant that the activity 
was done "sometimes/ and a value of 3 meant that the activity was done "regularly." 
The scale score for a given parent was computed as the average rating for all completed 
items in the scale-higher scores mean a higher level of reading or writing activities in the 
home. Thus, the highest possible scale score is 3.0, and the lowest possible scale score 
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is 1 .0. The reliability (internal consistency) of both scales as assessed in this study is 
quite reasonable: .80 for the reading activities scale and .71 for the writing activities 
scale. 



Effects on Reading and Writing Activities 

Exhibits 9.9 and 9.10 summarize data collected in the In-Depth Study for the Reading 
Activities Scale and the Writing Activities Scale. Parents in Even Start had a pretest 
mean of 2.2 points on the Reading Activities scale while parents in the control group had 
a pretest mean of 2.1 points. The standard deviation was .4 points. On the Writing 
Activities scale, parents in Even Start and in the control group had identical pretest mean 
scores of 1 .8 points, with a standard deviation of .4 points. 

Thus, parents in each of the two groups scored in the middle of the scale range on the 
pretest, indicating that they do not report particularly low levels of reading and writing 
activities in the home. While these pretest levels may be higher than expected, there still 
is room for growth on the posttest. However, Exhibits 9.9 and 9.10 also show that we 
do not see any significant change either in the Even Start group or in the control group 
between pretest and posttest. This means that we were not able to detect any 
measurable program effect over a nine-month period on the extent to which parents use 
reading and writing as literacy tools in the home. 



Conclusions About Effects on Parent Literacy Skills 

This evaluation measured effects on parent literacy skills in three areas: (1) functional 
literacy levels on a reading test, (2) the percentage of Even Start adults who obtained 
their GED certificate, and (3) reading and writing activities in the home. Even Start has 
had clear positive effects on GED attainment; the data are positive but mixed about 
effects on functional literacy; and there is no evidence that Even Start has changed 
reading and writing activities as reported by parents. 

Even Start projects may have been effective in improving the functional literacy of 
participating adults. Data from the NEIS show that adults who participate in Even Start 
achieve small, but positive gains on the CASAS Reading Survey, gains which are 
comparable in size to those observed in other studies of adult education programs. 
However, data from the In-Depth Study show that the gains of Even Start adults are not 
significantly greater than the gains achieved by a randomly assigned control group. 

It is important that both the NEIS and the In-Depth Study show that the amount of gain 
on the CASAS is directly related to the amount of instruction received through Even Start, 
This finding strengthens our confidence that the observed gains can be attributed to adult 
education. However, adults can and do participate in adult education outside of Even 
Start. The keys to enhancing program effects in this area may be to strive for the fullest 
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Exhibit 9.9 






Parent Reading Activities in the Home: 
Effects from the In-Depth Study 








Even Start 
(n=84) 


Control 
(n = 75) 




Mean 


S.D. 


Mean 


S.D. 


Pretest 

Posttest 

Gain 

Program Effect 
{Effect Size) 


2.2 
2.2 
0.0 


0.4 2.1 

0.4 2.1 

0.0 

0.0 points 
(.0 s.d.) 


0.4 
0.4 


• p < .05 










Exhibit reads: Even Start adults in the In-Depth Study averaged 2.2 points on Reading 
Activities in the home pretest. | 





Exhibit 9.10 






Parent Writing Activities in the Home: 




Effects from the In-Depth Study 






Even Start 


Control 




(n = 


84) 


(n = 


= 75) 




Mean 


S.D. 


Mean 


S.D. 


Pretest 


1.8 


0.3 


1.8 


0.4 


Posttest 


1.8 


0.4 


1.7 


0 3 


Gain 


0.0 




-0.1 




Program Effect 
(Effect Size) 




0.1 points 
(.25 s.d.) 




* p < .05 










Exhibit reads: Even Start adults in the In-Depth Study averaged 1 .8 points on Writing Activities 
in the home at pretest. 
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possible participation in adult education and for an increase in the number of hours per 
month. 

Even Start does have a positive effect on GED attainment. Data from both the In-Depth 
Study and the NEIS lead to the same conclusion in this area. Without Even Start, it is 
likely that few of these adults would have found the needed assistance to reach this goal. 
Compared to the total number of adults served, however, those that do attain a GED 
represent a relatively small percentage. One hypothesis is that a lack of retention and 
active participation in Even Start present major barriers to GED attainment. Most 
participants do not stay with the program long enough to overcome the lack of schooling, 
low English proficiency, poor test taking skills, or other reasons for not obtaining a 
diploma or GED. 

Still, some of the adults who had not attained a GED at the end of the reporting year for 
this analysis remain in Even Start and may, yet, achieve this goal. Data on this topic will 
be available for the evaluation's final report. Another observation is that the reporting 
system for the evaluation lacks information about individual or project goals relating to 
GED attainment, making it difficult to find the most appropriate basis for calculating 
attainment rates, GED attainment is not an immediate or appropriate goal for all 
participants, nor is it emphasized by all Even Start projects. 

Finally, we were not able to detect any measurable program effect over a nine-month 
period on the extent to which parents use reading and writing as literacy tools in the 
home. 
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Section Ten 

Effects on Parenting Skills 



This section presents findings from the NEIS and the In-Depth Study about Even Start's 
effects on parenting skills. We are using the term "parenting skills *o broadly include 
parents' knowledge about appropriate child behaviors and developmental needs as well 
as the activities and materials available in the home to foster children's development. 
Helping parents support their children's growth and development is one of the primary 
goals of Even Start, and the inclusion of paren^ng skills as a key program component is 
one of the features that distinguishes Even Start from other literacy and adult education 
programs. 

The section is divided into four parts: (1) parents' personal skills that may affect 
parenting, (2) the home learning environment, (3) parent-child reading behaviors, and (4) 
parents' expectations for their children. 



Personal Skills 

The parent interview for the In-Depth Study included two existing scales to assess 
parents' depression and sense of self-efficacy or locus of control. These measures were 
not part of the NEIS data collection. 



Description of the Measures 

Parents' personal skills have been found to be important mediating variables for parent- 
child interaction and parents' involvement in their children's education and development 
(Cleary, 1988; McLoyd, 1990; Parker et al., 1988). Parents who are depressed, have 
poor self-esteem, or fee! "out-of-control" are not likely to provide optimal conditions for 
their children's growth and development (Upshur, 1988). Thus, parental depression and 
self-efficacy were considered both as outcomes for the evaluation and as explanatory 
variables for other short-term. effects of Even Start. 

The Pearlin Mastery Scale was used to measure parents sense of self-efficacy or locus 
of control. This seven-item scale assesses the extent to which an individual regards one's 
life as being under one's own control rather than determined by fate. This scale is 
included in the nationaj evaluation of the JOBS program, and was selected over other 
scales because it includes a mix of positive and negative items and has high internal 
consistency (Cronbach's alpha of .81 in prior research and .63 for the In-Depth Study 
sample.) The author's work (Pearlin and Schooler, 1978) showing the relationship of the 
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Mastery Scale to stresses of parenting and family life provides evidence of the scale's 
construct validity. 



The scale consists of seven items such as "There is really no w«y I can solve some of the 
problems I have" and "I can do anything I really set my mind." The respondent is asked 
to rate each item on a four-point scale where "4" indicates "strongly agree" and "1" 
represents "strongly disagree". Scores were reversed for those items stated negatively, 
in order to create an average score where "4" indicates a sense of mastery. 

Parents' depression was assessed by the Center for Epidemiologic Studies Depression 
Scale-CES-D developed by Radloff (1977). This instrument is one of the most frequently 
used measures of depression cited in the psychological literature. The items were 
designed for use in general population surveys as a short self-report measure. The 20 
items on the scale, which represent a subset of those previously validated on longer 
depression scales, have high internal consistency as indicated by a Cronbach's alpha of 
.90 in previous research (Hall et al.. 1 985) and .89 in the In-Depth Study. Moderate test- 
retest correlations of .51 to .67 have been reported when the scale is administered after 
two to eight weeks (Radloff, 1977). The construct validity of the scale has been 
demonstrated by correlations of .49 to .85 with clinicians' ratings and correlations of .72 
to .84 with longer self-report scales used with psychiatric and general populations 
(Weissman et al., 1977). 

The 20 items on the CES-D include statements such as "I felt that everything I did was 
an effort," "I had crying spells," and "I enjoyed life." Respondents were asked to indicate 
how often they experienced each feeling during the past week. Items are rated on a four- 
point scale where "0" indicates "rarely or none of the time - less than one day" and "3" 
reflects "most or all of the time - 5-7 days". Positive items were reversed so that a total 
score could be computed, which ranges from 0 to 60. A total score of 16 or higher is 
considered to be indicative of depression (Hall et al., 1985). 



Effects on Personal Skills 

There were no significant program effects on parents' personal skills, although the 
prevalence of depressive symptoms among adults in both Even Start and the control 
group is noteworthy. 

Effects as Measured by the In-Depth Study. Exhibit 10.1 presents pretest and 
posttest scores on the Pearlin Mastery Scale and CES-D for Even Start and control 
parents. On the Pearlin Mastery Scale, the two groups are virtually identical at pretest 
and posttest. For the CES-D total score. Even Start and control parents also look quite 
similar at pretest and posttest. 

Exhibit 10.2 shows that 45 percent of Even Start and 39 percent of control parents hsd 
scores above IS indicating high depressive symptoms. By posttest, the proportions 
stayed the same for Even Start parents and rose slightly for the control parents. These 
slight differences are not statistically significant. However, it is interesting to note the 
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Exhibit 10.1 



Parents' Personal Skills: 
Results from the In-Depth Study 



Even Start 
(n = 84) 



Mean 



(s.d.) 



Control 
(n = 75) 



Mean 



(s.d.) 



Pearlin Mastery Scale* 

Pretest 

Posttest 

Gain 

Program Effect 
{Effect Size) 



CES-D Total" 
Pretest 
Posttest 
Gain 



Program Effect 
(Effect Size) 



2.9 
3.0 
0.1 



(0.6) 
(0.6) 

0.0 points 



2.9 
3.0 
0.1 



(0.6) 
(0.6) 



14.9 
16.1 
1.2 



(11.5) 
(10.9) 



0.2 points 
(.02 s.d.) 



15.0 
16.0 
1.0 



(11.3) 
(11.7) 



Exhibit reads: Even Start and control parents scored an average of 2.9 out of a possible 4.0 on 
the Pearlin Mastery Scale at pretest. 

'Score represents average across seven items rated on four-point scale, where "4" indicates more 
positive behaviors or attitudes. 

"Score represents frequency of 20 depressive symptoms rated on three-point scales. 





Exhibit 10.2 




Proportion of Parents with High Depressive Symptoms 
on CES-D: Results from the In-Depth Study 




Proportion of Adults 




Even Start 
(n = 84} 


Control 
(n = 75) 


Pretest 


45% 


39% 


Posttest 


45% 


42% 


Gain 


0% 


3% 


Program Effect 
(Effect Size) 


3% 
.08 


Exhibit reads; 45% of Even Start parents reported high depressive symptoms at pretest. 


•Percentages represent proportion of parents scoring 16 or higher on CES-D total 
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proportion of adults in both groups who report depressive symptoms. These results are 
similar to those reported by Hail and her colleagues (1985), who found that 48 percent 
of low-income mothers of young children had scores above the cut-off for the CES-D. 



Home Learning Environment 

The NEIS instruments and the In-Depth Study parent interview include several questions 
about parents' interactions with their children at home and the extent to which the home 
environment is conducive to young children's growth and development. 



Description of the Measures 

Information about parenting and the home environment comes from in-person parent 
interviews collected as part of the NEIS and the In-Depth Study. For the NEIS, these 
questions were drawn from three existing instrument: 

Home Observation for Measurement of the Environment (HOME). The 
HOME Inventory (Caldwell & Bradley, 1 984) has been used to measure the impact 
of parent training and education and to assess the quality of the home environment 
and mother-child relationship in the National Longitudinal Study of Youth (NLSY). 
Questions on reading activities and toys parents provide for learning were adapted 
for the Even Start evaluation. 

High/Scope Home Environment Scale (H/SHES). A highly structured parent 
interview was developed by the High/Scope Educational Research Foundation to 
assess parenting and parent-child interactions for the National Home Start 
evaluation (High/Scope Educational Research Foundation and Abt Associates Inc., 
1975). Questions were drawn from this instrument for Even Start in four 
categories of activities: books and reading, play materials available in the home, 
parent teaching, and participating in learning activities. 

Parent as a Teacher (PAAT). The Parent as a Teacher (PA AT) self-rating scale 
was developed by Strom (1 984) to assess parents' attitudes toward their children 
and to determine feelings and values concerning children's behaviors. This 
instrument was used in the evaluation of Project Giant Step (Layzer, Goodson, & 
Layzer, 1991) in New York City, a preschool program for disadvantaged families. 
For the NEIS, questions were selected from two subscales: play - understanding 
the developmental function of play and willingness to participate in play with the 
child; and teaching - understanding the learning process in young children and 
confidence in the parent's role as a teacher. For the In-Depth Study, two 
subscales were added in creativity parents' acceptance of creative functioning 
and encourage its development, and frustration - absence of frustration or irritation 
with child's demand for attention. 
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The parent interview developed for the In-Depth Study was purposefully designed to 
include the same questions as the NEIS while adding questions about parent-child 
activities outside of the home and family rules. These items were added because 
research with low-income families (e.g., Snow et al., 1991) suggests that homes 
associated with poor school performance often lack structure and supervision; in addition, 
children from low-achieving homes frequently lack the opportunities that middle-class 
children have for experiences and interactions outside of their families (Cochran and 
Brassard, 1979) that can afford opportunities tor learning. 

The home environment items on the parent interviews ask parents either to indicate the 
frequency of their own or their child's behaviors or to rate their agreement with 
statements about child development. In most cases, there are several behaviors or 
statements for each construct that we are interested in measuring. For example, the 
question about play materials in the home listed 1 2 toys or materials and asked the parent 
to indicate which ones are available in their home. For the analyses, composite scores 
calculated by taking an average across items were used rather than individual items, in 
order to handle missing data and increase the reliability of the scores. 

Exhibit 10.3 describes these composite scores and presents the internal consistency of 
the composite scores from the NEIS and the In-Depth Study data. The internal 
consistency, based on Cronbach's alpha coefficient, is a measure of the extent to which 
responses to individual items are correlated to the total score. Internal consistencies of 
,60 or .70 are considered to be moderate, and coefficients above .80 indicate a highly 
reliable composite score. In general, the reliabilities for the In-Depth Study composites 
are lower than those for the NEIS because of smaller samples of respondents. 
Nevertheless, most of the coefficients exceed .60, indicating moderate to good internal 
consistency. 

Two individual items about the home environment are also included in the parent 
interviews from the NEIS and the In-Depth Study. These include: 

Story reading: the frequency with which the parent reads to their child, rated 
on a five-point scale from never (1 ) to every day (5). 

Books in the home: the number of books in the home for the child to look at or 
read, categorized as none, one-two books, three-nine books, or more than ten 
books. 



Effects on Home Learning Environment 

Overall, data from the NEIS and In-Depth Study point to few significant program effects 
in this area. Data from the NEIS showed statistically significant gains on most of these 
measures; however, the effect sizes were generally small. In the In-Depth Study, program 
families reported gains on some measures, but gains were also seen among control group 
families. 
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Exhibit 10*3 

Description and Reliability of Home 
Environment Composite Scores 


Composite 
Score 


Description 


Response Choices 


Internal 
Consistency* 


NEIS 


IDS 


Learning 
activities 


Activities that can be used to 
teach or reinforce cognitive 
skills such as finding food on 
store shelves or sorting clean 
clothes (5 items) 


1 Never 

2 Once or twice 

3 On a regular basis 


.89 


.67 


Reading 
materials 


Reading materials found in the 
home (5 items) such as 
magazines and newspapers 


1 No 

2 Yes 


.71 


.50 


Play 

materials 


Materials found in the home 
that children can play with (12 
items) such as scissors, paints 
or puzzles 


1 No 

2 Yes 


.76 


.68 


Teaching 
child 


Things parent has helped child 
learn during the past month (7 
items) such as nursery rhymes 
and colors 


1 No, did not help 

2 Yes, helped 


.90 


.61 


Talk with 
child 


Topics parent have discussed 
with child (8 items) such as 
school activities or a television 
program 


1 Never 

2 Rarely, if ever 

3 Once/twice a month 

4 Once/twice a week 

5 Daily 


.91 


.65 


Family rules 


Extent to which family has 
rules for child's behavior (7 
items) such as amount of 
television or helping with 
chores 


1 Not at all like family 

2 Somewhat like family 

3 Very much like family 


NA 


.68 


Activities 
with child 


Parent-child activities outside 
of home in the past month (7 
iiciiio/ t>uun ds going xo a parK 
or grocery store 


1 Not at all 

2 Once or twice 

3 Once a week 

4 Several times a week 

5 Almost every day 


NA 


.49 


Parent as a 
Teacher 


Agreement with statements 
that reflect how children learn 
or the parent's role as a 
teacher (14 items on INJEIS; 28 
in IDS) 


1 Disagree strongly 

2 Disagree somewhat 

3 Agree somewhat 

4 Agree strongly 


.70 


.74 


'Coefficients (Cronbach's alpha) range from 0 to 1.0 and indicate the internal consistency of sec 
between individual items and composite scores. 


)re or retatk 


>nship 
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Effects as Measured in the In-Depth Study. Exhibits 10.4 through 10.11 
present the results from the In-Depth Study on the home environment measures. Even 
Start and control groups were similar at the pretest and posttest for the majority of these 
indicators* For example, on learning activities in the home (Exhibit 10.4) and teaching 
child at home (Exhibit 10.5) the Even Start and control groups each gained marginally 
from pretest to posttest. Even Start families had a statistically significant gain in the 
number of play materials in the home (Exhibit 10.6) from pretest to posttest; however, 
the control group showed a small gain over the same time period so that there is not a 
statistically significant program effect. 

There is a statistically significant program effect on the number of reading materials in the 
home (Exhibit 10.7). This variable measures the number of different types of reading 
materials in the home (e.g., books, magazines, newspapers). Even Start families showed 
a ten percent gain from pretest to posttest, which is a statistically significant difference. 
In contrast, control families reported only a two percent increase, which was not 
significant. These differences translate into a statistically significant program effect that 
corresponds to an effect size of .30 standard deviation units. 

Effects as Measured by the NEIS. Exhibit 10.12 presents the results on the home 
environment measures from the NEIS. The gain from pretest to posttest is statistically 
significant for all of these measures. This is partly due to the large number of adults in 
the NEIS database. When the effect sizes are considered, the gains range from small to 
moderate in size. The number of play materials in the home is the one indicator with a 
moderate effect size. 

In order to explore the relationship between gains seen on these parenting measures and 
participation in Even Start parent education, gains were calculated separately for four 
levels of parent education attendance, controlling for pretest scores on the parenting 
measures and the age of the child* The gains for adults who participated in more 
parenting education were significantly higher, but not importantly larger, than for parents 
with more minimal participation levels on all indicators except "learning activities" and 
"teaching child*" Exhibit 10.13 presents the gains for the variables where there was a 
statistically significant relationship to parenting education. The exhibit also shows that 
the effect sizes for these indicators were small to moderate for all except the "play 
materials" scale* Thus, while there is some evidence of a relationship between hours of 
instruction and growth on certain parenting measures, the results do not provide 
convincing evidence that these small gains can be attributed to the parenting education 
component of Even Start. 

Discussion of Effects on the Home Learning Environment. The Even Start 
effects we have observed on the home environment measures are small at best. There 
are several reasons why these findings may be weaker than in adult education or early 
childhood education. For one thing, there are few models to guide Even Start projects in 
developing effective instructional approaches to improving parenting skills. In addition, 
the data shown earlier in this report indicate that participants receive much less parenting 
education than literacy training. The research literature concurs that the outcomes of 
parenting education are particularly difficult to assess and there is little consensus about 
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Exhibit 10.4 






Learning Activities in the Home:* 
Results from the In-Depth Study 






Even Start 
(n = 84) 


Controi 
(n = 75) 




Mean || (s.d.) 


Mean | 


(s.d.) 


Pretest 


3.4 (0.7) 


3.3 


(0.7) 


Posttest 


3.5 (0.8) 


3.4 


(0.8) 


Gain 


0.1 


0.1 




Program Effect 
(Effect Size) 


0.0 points 




Exhibit reads: Even Start parents scored 3.4 out of a possible 5.0 on the Learning Activities scale 
at pretest. 


'Score represents average across five-point scales, where "5" indicates more positive home 
environment. 







Exhibit 10.5 










Teaching Child at Home:* 
Results from the In-Depth Study 












Even Start 
<n = 84) 


Control 
(n = 75} 






Mean 


(s.d.) 




Mean | 


(s.d.) 


Pretest 




73% 


(22) 




68% 


(22) 


Posttest 




74% 


(25) 




70% 


(21) 


Gain 




1% 






2% 




Program Effect 
(Effect Size) 






-1% 
(.05 s.d.) 






Exhibit reads: Even Start parents reported at pretest that they taught their children 73 percent of 
the skills included on the Teaching Child at Home Scale. 


•Score represents percentage of seven skills parents tried to teach child at home. 
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Exhibit 10.6 








Play Materials in the Home:* 
Results from the In-Depth Study 










Even Start 
(n = 84) 


Control 
(n m 75) 




Mean || 


(s.d.) 


Mean 


(s.d.) 


Pretest 


58% 


(18) 




55% 


(20) 


Posttest 


62% 


(22) 




57% 


(20) 


Gain 


4%* 






2% 




Program Effect 
(Effect Size) 




2% 
(.10 s.d.) 




Exhibit reads: Even Start parents reported at pretest that they had 58 percent of play materials 
listed on the Play Materials scale. 


•p < .05 












"Score represents percentage of 12 play materials available in home. 
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Reading Materials in the Home: a 
Results from the In-Depth Study 








Even Start 
(n = 84) 


Control 
(n = 75) 




Mean 


(s.d. 
) 


Mean 


(s.d. 
) 


Pretest 


55% 


(23) 


54% 


(27) 


Posttest 


65% 


(23) 


56% 


(27) 


Gain 


10%* 




2% 




Program Effect 
(Effect Size) 




8%* 
(.30 s.d.) 




Exhibit reads: Even Start parents reported at pretest having 55 percent of the reading materials 
listed on the Reading Materials scale. 


•p < .05. 










"Score represents percentage of five types of reading materials present in the home. 
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Exhibit 10.8 



Talking with Child at Home: 9 
Results from the In-Depth Study 





Even Start 


Control 




(n 


= 84) 


(n = 


75) 




Mean 


(s.d.) 


Mean 


(s.d.) 


Pretest 


3.8 


(0.6) 


3.7 


(0.8) 


Posttest 


3.7 


(0.8) 


3.7 


(0.8) 


Gain 


-0.1 




0.0 




Program Effect 




-0.1 




(Effect Size) 




(.13 s.d.) 





Exhibit reads: Even Start parents scored 3,8 out of 5,0 at pretest on the Talking with Child Scale. 

a Score represents average frequency across five-point scales, where "5" indicates daily 
conversations with child. 





Exhibit 10.9 




Family Rules:" 




Results from the In-Depth Study 






Even Start 


Control 




(n 


= 84) 


(n = 75) 




Mean 


(s.d.) 


Mean |j (s.d.) 


Pretest 


2.5 


(0.3) 


2.7 (0.4) 


Posttest 


2.6 


(0.4) 


2.7 (0.4) 


Gain 


0.1 




0.0 


Program Effect 




0.1 


(Effect Size) 




(.25 s.d.) 


Exhibit reads: Even Start parents scored 2.5 out of 3.0 at pretest on the Family mles scale. 


"Score represents average across seven three-point scales, where "3" 


indicates more positive 


behavior. 
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Exhibit 10.10 






Activities Outside Home:" 






Results from the In-Depth Study 








Even Start 


Control 




(n = 84) 


(n = 


75) 




Mean || (s.d.) 


Mean 


(s.d.) 


Pretest 


2.3 (0.5) 


2.2 


(0.5) 


Posttest 


2.3 (0.6) 


2.3 


(0.5) 


Gain 


0.0 


0.1 




Program Effect 


-0.1 




(Effect Size) 


(.20 s.d.) 




Exhibit reads: Even Start parents scored 2.3 out of 5.0 at pretest on Activities Outside of the 


Home. 








'Score represents average frequency across seven five-point scales where "5" indicates dailv 


activity. 









Exhibit 10.11 




Parent as a Teacher: 4 




Results from the In-Depth Study 






Even Start 


Control 




(n = 84) 


(n = 75) 




Mean || (s.d.) 


Mean ] (s.d.) 


Pretest 


2.9 (0.6) 


3.0 (0.5) 


Posttest 


3.1 (0.4) 


3.0 (0.4) 


Gain 


0.2 


0.0 


Program Effect 


.2 points 


(Effect Size) 


(.40 s.d.) 


Exhibit reads: Even Start parents scored 2.9 out of 4.0 on the Parent as a Teacher Scale. 


"Score represents average across 28 items rated on four-point scale, 


where "4" indicates more 


positive attitudes or behaviors. 
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Exhibit 10.12 






Home Learning Environment: 


Results from the NEIS 




Measure 


Number 

of 
Adults 


Average 
Pretest 


Standard 
Deviation 


Gain 


Effect 
Size 


Learning 
activities 


2,368 


2.2 


.49 


.18* 


.36 


Storv readinn 


3 221 


3 4 


1 2 


97* 


.22 


Books in home 


3,213 


3.4 


.86 


.22* 


.33 


Reading 
materials 


3,052 


.49 


.26 


.05* 


.19 


Play materials 


3,084 


.56 


.24 


.11* 


.46 


Teaching child 


3,027 


.60 


.22 


.06* 


.25 


Talk with child 


1,698 


4.1 


.84 


.17* 


.21 


Parent as a 
Teacher 


3,080 


3.3 


.39 


.05* 


.12 


Exhibit reads: Even Start parents scored 2.2 at pretest on the Learning Activities scale. 


| * p < .001 . 













the appropriate constructs and few psychometrically sound measures (Weiss & Jacobs, 
1988). 

A further consideration is that the high pretest averages on some of the measures leaves 
little opportunity to show growth. Parents in both the Even Start and control groups 
generally reported on the pretest that their child had participated in many learning 
activities at home, that there were many books in the home, and that they had done 
many things to help their child learn. It is possible that parents were trying to give 
socially acceptable answers and that the "right" answers to items were too transparent. 



Parent-Child Reading Task 

The Parent-Child Reading Task was designed specifically for the In-Depth Study to 
measure the parent-child interactions of Even Start participants and control families. The 
task consists of asking the parent to read a simple book to her child, while a trained 
observer uses a pre-coded rating form to record several aspects of parent-child 
interactions. The observation was developed to provide a direct measure of a shared 
literacy activity and parent-child interactions. 

in; 
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Exhibit 10.13 








Gains on Home Environment Measures in the NEIS 






by Hours of Parenting Education 






Measure 


Hours of 


Number of 


1 Averag 


A vera 


Effect 


Parenting 


Adults with 


e 


ge 


Size 




Education 


Matched 


Pretes 


Gain 








Scores 


t 






oiory reaoing 


1-10 hours 


472 


3.3 


.21 


.17 




1 1 -50 hours 


1 ,541 


3 4 


.26 


.21 




51-150 hours 


913 


3 5 


.28 


.23 




Over 150 


223 


3 5 


.43 


.35 




hours 










dooks in nome 


1-10 hours 


469 


3.3 


.12 


.14 




1 1 -50 hours 


1,535 


3.4 


.24 


.28 


• 


51-150 hours 


913 


3.4 


.23 


.27 




Over 1 50 


224 


3.5 


.32 


.37 




hours 










PdqW inn 

neauing 


1-10 hours 


441 


.50 


.01 


.04 


materials 


1 1 -50 hours 


1,456 


49 


.05 


.19 




51-150 hours 


879 


.48 


.07 


.27 




Over 1 50 


211 


.49 


.06 


.23 




hours 










rldy rnaLCridlS 


VI 0 hours 


435 


.63 


.10 


.42 




1 1 -50 hours 


1,471 


.59 


.10 


.42 




51-150 hours 


891 


.59 


.12 


.50 




Over 1 50 


221 


.57 


.14 


.58 




hours 










Tallr \Aiith nhilH 
lain Willi UililU 


1 -i u hours 


278 


4.2 


.1 1 


.13 




1 1 -50 hours 


770 


4.1 


1 0 
. I z. 


.14 




51-150 hours 


473 


4.0 


.28 


.33 




Over 1 50 


135- 


4.2 


.23 


.27 




hours 










Parent as a 


1-10 hours 


459 


3.3 


-.05 


-.13 


Teacher 


1 1 -50 hours 


1,481 


3.3 


.05 


.13 




51-150 hours 


895 


3.4 


.06 


.16 




Over 1 50 


215 


3.3 


.16 


.41 




hours 










Exhibit reads: Parents with more than 1 50 hours of parenting education scored 3,5 on the story readina 


item, compared with parents with 1-10 hours of parent education who scored 3.3. 




* p < .001. 
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Description of the Measure 



The task is based on research that stresses the importar ) of reading books to the 
literacy development of young children. Findings from various studies over the last two 
decades note that middle-class children who are read to understand and know more 
words (Chomsky, 1972), have increased language and reading growth (Ninio and Bruner, 
1978), learn basic concepts about books (Sulzby, 1985; Snow and Ninio, 1986), and 
benefit additionally because adults' reading to children acts as "scaffolds" in emerging 
literacy development (Edwards, 1 989). Other research documents the importance of the 
methods used by parents when they read aloud to their children. The manner in which 
parents read to children can inhibit or facilitate the child's interest and skill in reading 
(Lancy, 1988). The quality of the interaction surrounding parent-child reading has been 
found to shape early reading development more than merely the presence of books or 
storytime routine (Mason and Allen, 1986). 

Further, parents' questioning strategies used when reading aloud to children may promote 
or limit the cognitive benefits of story book reading (Pellegrini et al., 1990). Heath (1983) 
has written of the importance that questioning plays in the development of school-type 
literacies in young children. In three communities she studied, only parents in the middle- 
class sample used the kind of cognitive questioning strategies that teachers commonly 
use in school-based instruction. She considers children who are not familiar with this 
cognitive approach to be at risk when confronted with the demands of formal schooling. 
Her sample included poor black families from a rural southern region, not unlike some of 
the Even Start sites. 

Thus, the research indicates that parents teach children through the medium of story book 
reading as they question and. comment on the text and pictures, and as they initiate and 
respond to the child's comments. The management of the story book session reveals 
much about the relationship, interaction, and teaching strategies that occur between the 
parent and child around a pleasurable task which, if effectively carried out, promotes 
literacy development. Several criteria were used to select the book for this task: 

♦ The book had to have a story line. Previous research indicates reading 
stories has stronger associations with children's reading and language 
development than looking at books or reading wordless picture books, 
alphabet books or nursery rhymes (Wells, 1985). 

• The book had to have available in Spanish and English to ensure 
comparability of subject ano difficulty of the story across language groups. 

• The book had to have relatively easy vocabulary so that it is not too difficult 
for low-literate parents. 

♦ The book had to have based on subject matter appropriate for girls and 
boys, children from diverse cultures, and children living in urban and rural 
areas. 
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A book entitled Three Billy Goats Gruff was selected that met all of the above criteria. 

Parents were asked to read the simple story to their child during the testing/interview 
session for the In-Depth Study. The choice of the English or Spanish version of the books 
was left up to the parent. The inte-viewer told the parent that we were interested in 
learning how parents and children read together, and instructed the parent to read the 
book the way she usually would read to her child. The interviewer rated the parent's and 
child's behavior as they read the book. 

The rating scale was developed to record parent-child interactions and the parent's 
approach to reading during the brief story book reading session. The rating scale is 
divided into two parts. On Part I, the interviewer scored the way the parent read the 
book (e.g., labeling pictures, asking questions of the child) as well as the child's response 
and behavior during the session (e.g., pointed to pictures, responded to parent's 
questions). These categories are adapted from a rating scale developed by Resnick and 
his colleagues (1987), based on videotapes of mothers reading to their young children, 
and modified by Edwards {1 989). The items are scored as absent or present; counts of 
behavior were not recorded because pilot testing of the instrument indicated it was not 
feasibles to categorize and keep a count of diverse behaviors during these relatively brief 
observations. 

Part II of the coding scheme was filled out by the interviewer at the completion of the 
parent-child reading activity. These general ratings of the quality of the parent-child 
interaction and the parent's reading are adapted from work conducted by Lancy and 
Draper (1988). This section includes items rated on a three-point scale to describe the 
reciprocity and quality of parent-child interaction. Parent and child were rated separately 
on items such as physical contact and task engagement. 

The interviewers were trained by using videotapes of several parent-child dyads reading 
Three Billy Goats Gruff. In addition, an administration manual provided a description and 
examples of each coding category. For the analysis, composite scores were created as 
more reliable indicators. Based on conceptual groupings of parents' behaviors assessed 
on Part I, two composite scores were created and then evaluated for internal consistency. 
Additional composite scores were created by aggregating child behaviors and the general 
affect scores. The composite scores from Part I of the rating scheme were created by 
totaling the number of different behaviors that occurred during the session. On Part II, 
composite scores were calculated as the average scoring across the three-point scales. 
Exhibit 10.14 presents the reliability coefficients for the composite scores and lists the 
individual items included. Reliabilities are quite good, ranging from .71 to .80 for the five 
composite scores. 



Effects on the Parent-Child Reading Task 

There were no significant gains for any of the indicators of parents' behaviors in either 
Even Start or the control group. Significant gains from pretest to posttest were seen for 
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Even Start children's involvement in the reading experience, but no program effects were 
found. 



Effects as Measured by the In-Depth Study. The pretest and posttest means 
for the behavioral ratings on the Parent-Chiid Reading Task are presented in Exhibit 10.1 5. 
There was virtually no change between pretest and posttest for either the Even Start or 
control groups, and no program effects. Similarly, there are no significant program efforts 
for the general affect ratings of parents or children {Exhibit 10.16), although Even Start 
children did show significant increases in their involvement and interaction during the 
reading activity. 



Parents' Expectations 

Description of the Measures 

Questions about parents' expectations for their children's school success were adapted 
from the parent interview designed by Abt Associates for use in Prospects, the national 
longitudinal study of Chapter 1 (Puma et al„ 1 993). In both the NEIS and In-Depth Study, 
parents were asked to rate how well they expected their child to do in school (from "very 
weir to "very poorly") and the likelihood that their child will graduate high school (from 
"very likely" to "probably won't graduate high school"). 



Effects on Parents 9 Expectations 

The NEIS and In-Depth Study data showed significant gains and similar results for 
parents' expectations about their children's school success. 

Effects as Measured by the In-Depth Study. Exhibit 1 0.1 7 presents 
expectations among parents in Even Start and the control group. There is a small, but 
statistically significant program effect on parents' expectations for their children's success 
in school. Although both groups of parents tended to believe their children will do well 
in school, the expectations of parents in Even Start increased from pretest to posttest 
while those of the control parents stayed the same. 

Parents in both Even Start snd the control group, on average, felt that their children are 
likely to graduate from high school. As Exhibit 10.17 indicates, there are no statistically 
significant differences from pretest to posttest for either group, and no program effect. 

Effects as Measured by the NEIS. The results from the NEIS are presented in 
Exhibit 10.18. Interestingly, the average expectations and the magnitude of the gain are 
similar to those seen in the IDS. Given the sample sizes here, these small gains are 
statistically significant for the NEIS data. 
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Exhibit 10.14 

Composition and Reliability of Scores on Parent-Child Reading 

Task 


Composite Score 


Items 


Internal 
Consistency 

vwCI 1 lv»lt2Itlo 


Parent describes 
book to child* 


Points to pictures or words; labels 
pictures; describe pictures or text; 
elaborates on pictures or text. 


.71 


Parent questions or 
responds to child* 


Links pictures or story to child's life; 
asks child to identify pictures; repeats 
child's comments or words; 
elaborates on child's comments; 
responds to child's questions; asks 
comparative inferential or cause-and 
effect questions. 


.71 


Child responds* 


Points to pictures; labels or names 
picture; repeat words or elements of 
story; acts out or makes sound related 
to story; links story content or 
pictures to own life; responds to 
oarent's Questions* a<%k<% nup«5tion<5 
makes comments related to book or 
parent's comments. 


.74 


Parent's general 
affect b 


Task engagement; positive affect; 
interactive contact with child; physical 
contact with child; pace; control of 
book; reading fluency; reading 
delivery. 


.80 


Child's general 
affect b 


Task engagement; positive affect; 
interactive contact with parent; 
physical contact with parent. 


.75 


*Each item in composite scored as present or absent. 
b Scored on three-point scale. 
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Exhibit 10.15 






Parent-Child Reading Behaviors: 
Results from the In-Depth Study 








Even Start 
(n = 84) 


Control 
(n m 75) 




Mean j| 


(s.d.) 


Mean 


(s.d.) 


Parent describes bcok* 










Pretest 


2.1 


(1.4) 


2.4 


(1.4) 


Posttest 


2 1 


(1.3) 


2.3 


I 1 


Gain 


0.0 




-0.1 




rTOyiam CTTeCl 

(Effect Size) 




0.1 points 
(.07 s.d.) 




Parent questions or responds 
to child b 










Pretest 


1.6 


(1.7) 


1.9 


(1.7) 


Posttest 


1.6 


(1.6) 


1.8 


(1.6) 


Gain 


0.0 




-0.1 




Program Effect 
(Effect Size) 




0.1 points 
(.06 s.d.) 




Child's responds 0 










Pretest 


2.6 


(2.0) 


3.0 


(2.0) 


Posttest 


2.9 


(1.9) 


3.0 


(1.8) 


Gain 


0.3 




0.0 




Program Effect 
(Effect Size) 




0.3 points 
(.15 s.d.) 




Exhibit reads: Even Start parents at pretest exhibited an average of 2.1 out of 4 behaviors related 

to describing the book to their child. 

•Score represents occurrence of four parent behaviors. 

b Score represents occurrence of six parent behaviors. 

c Score represents occurrence of seven child behaviors. 

♦p<.05. 
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Conclusions about Effects on Parenting Skills 



This evaluation measured effects on parenting skills in four areas: (1) parents' personal 
skills, (2) the home learning environment, (3) parent-child reading, and (4) parents' 
expectations for their children. In general, there were few program effects on these 
measures. 





Exhibit 10.16 






General Affect Ratings on Parent-Child Reading Tasks: 
Results from the In-Depth Study 






Even Start 
(n = 84) 


Control 
(n = 75) 




Mean 


(s.d.) 


Mean 


(s.d.) 


Parent's general affect" 










Pretest 


2.3 


(0.4) 


2.4 


(0.4) 


Posttest 


2.3 


(0.0) 


2.4 


(0.4) 


Gain 


0.0 




0.0 




Program Effect 
(Effect Size) 




0.0 points 




Child's general affect" 










Pretest 


2.1 


(0.5) 


2.3 


(0.5) 


Posttest 


2.3 


(0.5) 


2.4 


(0.5) 


Gain 


0.2* 




0.1 




Program Effect 
(Effect Size) 




0.1 points 
(.14 s.d.) 




Exhibit reads: Even Start parents at pretest received an average rating of 2.3 out of 3 on the 
general affect hems. 

'Score represents average on eight items rated on three-point scales, where "3" indicates positive 
behavior. 

"Score represents average on four items rated on three-point scales, where "3" indicates positive 
behavior. 


•p < .05. 











Parents' personal sense of well-being, as measured by a sense of mastery and lack of 
depression, has been cited in the research literature as related to the nature and quality 
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of parent-child and family relationships. Data from the In-Depth Study did not reveal 
significant differences between program and control groups. The overall prevalence of 
depressive symptoms was high among this group of families, although similar to those 
reported with other low-income populations. It is possible that these psychological 
problems are difficult to ameliorate with programs such as Even Start, or at least difficult 
to change in the short-term. Perhaps with continued gains in other areas such as 
educational attainment and employment, parents will express different opinions about 
their sense of mastery and happiness. It is also possible that projects have not focused 
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Parent's Expectations for Their Child: 
Results from the In-Depth Study 






Even Start 
(n = 84) 


Control 
(n = 75) 




Mean || (s.d.) 


Mean j 


(s.d.) 


How child will do in school 3 








Pretest 


4.3 (.82) 


4.0 


(.84) 


Posttest 


4.4 (.69) 


4.0 


(.96) 


Gain 


0.1* 


0.0 




Program Effect 
(Effect Size) 


.10 points* 
(.12 s.d.) 




Likelihood of child graduating 
from high school 6 








Pretest 


3.7 (.62) 


3.6 


(.68) 


Posttest 


3.8 (.43) 


3.7 


(.45) 


Gain 


.1 


.11 




Program Effect 
(Effect Size) 


-.02 points 
(.03 s.d.) 




Exhibit reads: Even Start parents at pretest rated the likelihood that their child would graduate 
from high school as 3.7 out of a possible 4.0. 

•Item rated on five-point scale where "5" equals "very well" and "1" indicates "very poorly 1 *. 
b ltem rated on four-point scale where "4" indicates "very likely" and "1" indicates "probably will 
not graduate from high school". 


*p < .05. 
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on thar-o problems and may be able to help parents through increased collaboration with 
r and mental health agencies. 

Even Start appears to be having a significant impact in only one area of the home 
environment: the number of reading materials in the home. While all variables on the 
NEIS showed a significant change from pretest to posttest but small effect sizes, this was 
the one variable where there was a significant gain among Even Start participants in the 
In-Depth Study. Since this is one measure out of many, this finding should be interpreted 
with caution. However, many Even Start projects make a concerted effort to increase the 
number of books, magazines and newspapers in participants' homes, either through loans 
or free donations of reading materials. 

Although parents' expectations for their children are high at pretest, there were small 
program effects on parents' expectations for school success which were seen in the NEIS 
and In-Depth Study data. 

There were virtually no significant program effects seen in the parent-child reading task. 
The Parent-Child Reading Task was developed as a new measure to extend the 
information collected from the In-Depth Study beyond the traditional paper-and-pencil 
tests. In addition. Even Start project directors urged us to collect some direct assessment 
of parent-child interaction and a shared literacy activity. However, the limited changes 
recorded from pretest to posttest on this measure aise questions about the utility of the 
rating scale in its present format. A third round of data on the Parent-Child Reading Task 
will be available this summer and included in the final report of this evaluation. At that 
time, we will have a better sense of the value of the rating scale and recommendations 
for revisions to the form for future research. 

One shortcoming of the measure may be that it is not sensitive enough to differences 
between parents to show changes over time. The choice to make the coding a simple 
check of behaviors rather than a count means that we cannot differentiate parents who 
display multiple examples of positive behaviors from those who exhibit the behavior only 
once. Much of the prior research in this area used videotapes and coded mother-child 
behaviors from the tapes. We felt that this would be a costly approach and also would 
make some parents and children feel uncomfortable. 

Another difference between the current measure and the research tools on which it was 
based is the purpose of the observation. Most of the prior research studies {e.g., Lancy 
and Draper, 1988; Pellegrini et al., 1990; Resnick et al., 1987) were attempting to 
describe the content and quality of parent-child reading in order to determine whether 
mothers incorporated teaching strategies when they read to their child or to relate 
parents' reading strategies to the acquisition of the child's reading ability. Thus, using 
this type of instrument to assess program effects is a new application that may require 
more differentiated coding schemes. 



Even Start 10-21 1 J ° Effects on Parenting Skills 





Exhibit 10.18 








Parent's Expectations for their Child: 
Results from the NEIS 






Measure 


Number 

of 
Adults 


Averag 
e 

Pretes 
t 


Standar 
d 

Deviati 
on 


Gain 


Effect 
Size 


How child will do in 
school* 


2,857 


4.2 


.81 


.10* 


.12 


Likelihood of child 
graduating from 
high school 6 


3,098 


3.7 


.53 


.05* 


.10 


Exhibit reads: Parents are pretest rated the likelihood that their child will graduate from high school 
as 3.7 out of a possible 4. 

•Item rated on five-point scale where "5" equals "very well" and "1" indicates "very poorly". 


b ltem rated on four-point scale where "4" indicates "very likely" and "1" indicates "probably will 
not graduate from high school. 


# p < .05. 
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Section Eleven 

Effects on Families 



This section presents results from the In-Depth Study on the social supports and financial 
resources available to Even Start families. Information on employment status from the 
NEIS and in-Depth Study also is reported. By providing support services to pas ticipating 
families and engaging parents in program activities, it is hypothesized that Even Start may 
help families develop a wider sociH network and greater access to social services. In 
addition, aduit education services could enable parents to find work or change to higher 
paying jobs, with the long-term effect of increasing income from wages and reducing 
reliance on public assistance. In addition, this section describes results from focus groups 
with Even Start parents about their perceptions of the program's impact on their families. 



Social Support 

The term "socia; support" refers to the help and support offered to individuals and families 
by their relatives, friends and neighbors. The availability of this support can affect one's 
psychological well-being by providing direct assistance as well as serving as an informal 
referral source to community services (Gottlieb, 1 976). Adequate social support also has 
been linked to outcomes for children. The availability of a support network for parents 
influences children both in terms of the amount of emotional energy that parents have for 
their children as well as the increased opportunity for interactions and experiences outside 
of the home that provide sources of cognitive and social stimulation for children (Cochran 
and Brassard, 1979). 



Description of the Measure 

The Inventory of Socially Supportive Behaviors (ISSB), developed by Barrera, was included 
in the parent interview for the In-Depth Study to obtain information about parents' social 
supports. This measure was chosen over other measures of social support because it 
includes concrete behaviors in additional to emotional support, uses clear and simple 
language, and assesses the frequency of support rather than perceived satisfaction with 
available supports. 

The ISSB was designed to assess various types of assistance that people have available 
in everyday life. The scale includes 40 behaviors such as: provided you with a place 
where you could get away for a while, provided you with transportation, told you who 
you should see for assistance, and loaned you over $25. Respondents are asked to rate 
the frequency of each event in the past month on a five-point scale from "not at air (1) 
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to "almost every day" (5). The measure is scored by creating either a total additive score 
across all items or an average score if ratings are missing from some items. 

In previous research, the measure has demonstrated adequate psychometric properties. 
The test-retest reliability for the total score was .88 when the measure was administered 
twice over a two-day period (Barren., 1981). The internal consistency of the total score 
has generally been above .90 (Barrera, no date). For the In-Depth Study, the ISSB scale 
was shortened to 27 items to reduce respondent burden. The internal consistency, as 
measured by Cronbach's alpha, was .92 for the total score based on these 27 items. 



Effects on Social Support 

There was no program effect on the social support scale in the In-Depth Study. This 
scale was not part of the NEIS. Exhibit 11.1 presents the pretest and posttest scores on 
the ISSB. Parents in Even Start and the control group gave very similar ratings to the 
frequency of social support, and there are no statistically significant differences between 
the groups. 



Family Resources 

A number of researchers have discussed the impact of limited financial resources on 
family dynamics, parenting, and child growth and development. Families living in poverty 
are more likely to experience greater and more chronic stress than middle or upper income 
families; in addition, the incidence of major stresses, such as inadequate housing and lack 
of money for food, creates psychological stress that affects parent-child and parent-parent 
interactions within the family (Parker et al., 1988). As a result, poor families are more 
likely to exhibit punitive parenting styles than more economically advantaged families 
(McLoyd, 1990). 



Description of the Measures 

Information about family resources comes from three areas of the parent interview: a 
question about the major sources of household income, a question categorizing the level 
of family income, and a set of questions about the adequacy of family resources. 

The family resource scale is an adaptation of an instrument developed by Leet and Dunst 
for use in family intervention programs (Dunst et al., 1988). It is one of the few 
measures we have seen that tries to assess the adequacy of resources for basic 
necessities (e.g., food for two meals a day, indoor plumbing, and owning or having access 
to a telephone). The scale includes 25 items which respondents are asked to rate on a 
five-point scale from "not at all adequate" (1) to "almost always adequate" (5). An 
average rating across the 25 items was created as a composite score. 
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Exhibit 11.1 

Social Support: Results from the In-Depth Study 




Even Start 
(n=84> 


Control 
(n=75) 


Mean || (s.d) 


Mean 


(s.d) 


Inventory of Socially Supportive 
Behavior* 

Pretest 2.4 (0.7) 2.2 
Posttest 2.4 (0.8) 2.4 
Gain 0.0 0.2 

Program Effect -0.2 points 
(Effect size) (.25 s.d.) 


(0.8) 
(0.8) 


Exhibit reads: Even Start parents scored an average of 2.4 out of 5.0 at pretest on the ISSB scale. 
'Score represents average across 27 items rated on five-point scale, where n S m indicates greater 
frequency of social support. 



In prior research, the measure has shown adequate psychometric properties. The authors 
report a Cronbach's alpha of .92, a split-half reliability of .95 corrected for length using 
the Spearman-Brown formula, and a test-retest reliability of .52, based on administering 
the scale to 45 individuals on two occasions two months apart (Dunst and Leet, 1987). 
In the In-Depth Study, the internal consistency of the composite score was .85. 



Effects on Family Resources 

There was a statistically significant increase in the adequacy of family resources reported 
by Even Start families; however, control group families also gained a small amount and 
there was no significant program effect. There were no program effects on income or 
reliance on government assistance. The lack of effects on income is to be expected, 
since changing income is a relatively long-term outcome for Even Start. 

Exhibit 1 1.2 presents the results on the Family Resource Scale for Even Start and the 
control group. Both groups reported an average of 3.6 out of a possible 5.0 on the 
pretest, indicating that they perceived their resources to be between "sometimes 
adequate" and "usually adequate" at the time they entered Even Start. This suggests that 
parents did not perceive a large problem with a lack of resources. In both groups, the 
average rating of the adequacy of family resources rose slightly from pretest to posttest. 
This gain was statistically significant for the Even Start participants; however, the 
difference was not large enough when compared to the gain among the control group to 
show a statistically significant program effect. 
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Exhibit 11.2 






Adequacy of Family Resources: Results from the In-Depth Study 






Even Start 
(n = 84) 


Control 
(n = 75) 




Mean || (s.d) 


Mean 


(s.d) 


Family Resource Scale* 








Pretest 

Posttest 

Gain 


3.6 (0.6) 
3.8 (0.6) 
0.2* 


3.6 
3.7 
0.1 


(0.6) . 
(0.5) 


Program Effect 
(Effect size) 


0.1 points 
(.17 s.d.) 




Exhibit reads: Even Start families scored an average of 3.6 out of 5.0 at pretest on the Family 
Resources Scale. 

"Score represents average across 25 items rated on five-point scale, where "5" indicates more adequate 
resources. 


*p<.05. 









Exhibit 11.3 

Financial Resources: Results from the In-Depth Study 




Even Start 






Control 




(n = 84) 






(n = 75) 


Household Income Less than $10,000 










Pretest 


72% 






72% 


Posttest 


62% 






71% 


Gain 


10% 






1% 


Program Effect 




9% 




Receiving Government Assistance 










Pretest 


48% 






41% 


Posttest 


44% 






43% 


Gain 


4% 






-2% 


Program Effect 




6° 


/o 




Exhibit reads: 72 percent of Even Start parents had household incomes less than $10,000 at pretest 



Exhibit 11.3 shows the percentage of families at pretest and posttest (a) whose 
household income was less than $10,000, and (b) who receive government assistance. 



EvanStmrt 77-* Effects on FmmMms 



:RIC 



While the percentage of Even Start families with income below $10,000 dropped from 
72 percent at pretest to 62 percent at posttest, this is not a statistically significant 
difference from the changes seen among the control group. 

A second way to analyze these data is to focus just on those families whose income was 
below $10,000 at the pretest to see how many were able to increase their income above 
this level. Exhibit 1 1 .4 shows that 23 percent of Even Start families and 13 percent of 
control families reported increases that moved their income from under to over $10,000. 
However, these differences are not statistically significant, 

The percentage of Even Start families on government assistance dropped from 48 percent 
at pretest to 44 percent at posttest (see Exhibit 1 1 .3), while the percentages among the 
control group rose slightly. However, these differences are not statistically significant. 
Exhibit 1 1 .5 shows the percentage of families in each group who were able to get off of 
public assistance in Even Start (24 percent) and the control group (16 percent). Once 
again, these differences are not statistically significant. 



Employment Status 

Although Even Start is not primarily an employment and training program, the adult 
education component of the program could effect changes in parents' employment status 
by increasing their skill levels and employability. 



Description of the Measure 

The parent interview for the NEIS and the In-Depth Study asks respondents to indicate 
their current employment status as working full-time, working part-time or not working. 
For the NEIS, these questions were asked at program entry and again either at the end 
of the program year or at exit. In the In-Depth Study, employment status was asked at 
the pretest and posttest. The information was specific to the adult targeted for the 
evaluation, and does not necessarily reflect whether anyone in the household is employed. 

The analyses focused on the percentage of adults who were unemployed at program entry 
versus those working part-time or full-time. We aggregated data about part-time and full- 
time employment for a number of reasons. First, Even Start project directors pointed out 
that for mothers of young children, part-time work is a positive outcome. Also, part-time 
employment in a job with higher wages can be an improvement over a full-time but low- 
paying job. 



Even Stsrt 



161 

11-5 Effects on FamSms 



Exhibit 11.4: Percentage of Families Who Increased Income 
Above $10,000 by Posttest: In-Depth Study 



i 




Exhibit reads: Of those families whose household income was less than $1 0,000 at pretest, 23 percent of Even 
Start families and 13 percent of control families reported incomes above $10,000 at posttest. 



Effects on Employment Status 

Based on results from the In-Depth Study and the NEIS, Even Start does not appear to 
have any significant short-term effects on participants' employment status. However, the 
conceptual model for this evaluation shows that effects on employment would be 
expected in the long-run, not over a short time period. 

Effects as Measured by the In-Depth Study. Exhibit 1 1 .6 shows the percentage of 
parents in the Even Start and control groups who were not employed at the pretest and 
posttest. While some adults who were unemployed at pretest became employed by 
posttest, there are no significant program effects. When the sample is restricted to just 
those adults who were not employed at pretest, approximately equal percentages of 
adults in Even Start (12 percent) and the control group (15 percent) found work by 
posttest (Exhibit 1 1.7). 

Effects as Measured by the NEIS. Exhibit 1 1 .8 presents information about the percentage 
of adults who were not employed at program entry for the total group of respondents and 
for subgroups of participants. Approximately 78 percent of the adults participating in 
Even Start were not employed at pretest. Nearly 10 percent of these participants found 
employment by the end of the program year or by the time that they exited from Even 
Start. The results were fairly consistent across subgroups of participants. 
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Exhibit 1 1 .5: Percentage of Families Who Got Off Public 
Assistance as Primary Source of Income 
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Exhibit reads: Of those famflies whose primary source of income was public assistance, 24 percent of Even 
Start families and 16 percent of control families no longer relied solely on public assistance by posttest. 



Parents' Perceptions of Program Impacts 

As part of the In-Depth Study, parents in Even Start were asked for their assessment of 
the program's impact. 



Description of the Measure 

During the annual site visits to the ten In-Depth Study projects, Abt staff conducted focus 
groups with Even Start parents to obtain their views of the program. These discussions 
were held with approximately ten parents at each site. Project directors invited parents 
to take part in the focus group, but their participation was voluntary. Topics for the focus 
groups included the reasons why parents joined Even Start and the impact of the program 
on themselves and their family. 



Perceived Effects on Adults 

Parents described a number of positive effects of their participation in Even Start, 
including improved personal skills, increased parenting ability and learning gains. 
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Exhibit 11.6 




Employment Status: Results from the In-Depth Study 




Even Start 


Control 




I (n = 84) 


<n = 75) 


Not Employed 






Pretest 


83% 


80% 


Posttest 


76% 


76% 


Gain 


7% 


4% 


Program Effect 


3% 


Exhibit reads: 83 percent of Even Start parents were not employed either part-time or full-time at 


pretest. 







In some programs parents noted that the main reason they joined Even Start was for the 
adult education. Some parents were interested in preparing for the GED certificate, 
continuing their education or improving English proficiency. Others stated that education 
was a way to improve their lives. As one woman said, "I want to better myself, to take 
up a trade, to be a nurse's aide." Another parent indicated a desire to "do something 
with my life." 

Several parents commented that having the early childhood services available was a plus 
for them as well as their children. Being able to bring their children with them to adult 
education classes is important to a number of mothers. For some it is the convenience, 
for others it is the safety-knowing where their children are and being able to check on 
them during the day eased parents' concerns. 

Even Start participants also discussed the changes they experienced in their interactions 
with their children. Although most parents indicated that the parenting classes were not 
the main reason for joining the program, they now viewed this component as extremely 
important. Through the parenting classes and home visits, parents described being able 
to communicate more effectively with their children. The program also has helped them 
learn how to play with their children--to be creative with their children, to make games 
for their children, and to plan activities for their children at home. 

Parents described being able to interact more positively with their children and deal with 
the stresses they feel as a parent without "exploding." Several parents spoke of having 
more patience with their children and using less physical punishment. One mother told 
how she has learned not to criticize her children as much, but rather to give them praise; 
further, she felt that she has developed more realistic expectations about her children and 
herself. 
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Exhibit 1 1 .7: Percentage of Adults Not Employed at 
Pretest who Found Employment by Posttest 
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Exhibit reads: Of those adufcs who were not employed at pretest, 12 percent of adults in Even Start and 15 
percent of adults in the control group were employed at posttest. 

A number of parents commented that they have higher personal goals and more self- 
esteem as a result of Even Start. One woman stated that before she joined the program, 
she viewed the GED as "working papers," but now she sees the certificate as "the first 
step towards a better life." 

The program also has reduced the parents' sense of isolation and being "trapped at home 
with their children." Although some of the women knew each other before enrolling in 
the program, they had only limited interaction. As one mother said, "you can live right 
next door to someone and not know them." Now t;iey check up on each other and visit 
or call if someone has missed class. Another mother commented that "for most of us, 
it's just us and our kids." Even Start has given them the opportunity to interact with 
other adults who have similar problems and experiences. As a result, participants feel 
better able to cope with life situations. 

In several programs, participants described Even Start as their "family." In one project, 
parents gave staff high marks for being respectful and willing to do anything for the 
families--"we just pick up the phone and tell them we need help and they are there." In 
another site, parents told how staff treat them "as human beings without putting you 
down or judging you," but all the while "helping U3 in a new way of life." In a third site, 
parents pointed out that the friendly and supportive environment of the program allows 
parents to feel valued and know that their concerns are taken seriously. 
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Perceived Effects on Children 



Parents were quite positive about Even Start's effects on their children. Children's 
attitudes toward reading and schooling have changed to excitement and eagerness to 
learn. Several parents cited specific skills that their children have learned, such as using 
scissors, learning the alphabet and the name of colors. Parents with older children stated 
that their younger children in Even Start are much better prepared for public school than 
their older s.bl.ngs-they know how to take the bus, they can read, they speak English 
better and they are "just less afraid." 

In addition to cognitive skills, improved verbal and social skills were documented by many 
parents. One mother stated that her child rarely spoke before he attended Even Start, and 
"now he talks constantly." Others commented that the program has helped their children 
to be less shy, to speak more clearly, and to separate more easily from their mothers. As 
one mother described it, " my children does more for himself in everything-in eating, in 
dressing himself.. .he isn't always hanging onto me anymore." 

Parents reported that their children are now better behaved. One mother described her 
son's pre-Even Start activity level as similar to "a grasshopper," but now he is more calm 
and controlled. Parents also described their children as more helpful and cooperative at 
home. The organizational skills that children learn at school (e.g., putting things back in 
their place) have carried over to the home as well. 



Conclusions about Effects on Families 

The In-Depth Study included several measures of the effects of Even Start on participating 
families, including their perceived social support, adequacy of financial resources, income 
level and sources and employment status. The NEIS instruments had questions about 
income and employment. Across these measures, gains from program entry to the end 
of the first program yeyr were minimal. It is possible that these areas are difficult to 
change because they are affected by circumstances beyond the control of the program, 
such as the local economy and the availability of jobs. It is also likely that these indices 
will not show large changes in the short-run, but instead require longer interventions and 
other positive short-term impacts in order to achieve significant gains. 

While there are few effects for families on these quantifiable variables. Even Start 
participants quite a number of qualitative changes in their lives and the lives of their 
children. Listening to the personal stories of program participants, it becomes apparent 
that most of the changes in attitudes and skills that the parents see in themselves and 
their children pre positive short-term impacts of the program. Parents describe 
themselves as moving toward their goals of an educational certificate, getting a job and 
being a better parent. Given the current status of the Even Start parents, it is reasonable 
that we do not yet see changes in the more distal outcomes of increased employment and 
income. 
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Exhibit 11.8 








Employment Status: Results from the NEIS 








All Participants 


Not Employed at htake 


Group 


1 OlOl 

Adults 


Not Employed at 
Intake 


Total 


Found 
Emplovment 


M 


% 


Adults 


N 


% 


Age at Intake 














Unaer Z\ 


1.131 


995 


88.0 


916 


65 


7.1 


21-25 


3,029 


2,522 


83.3 


2,320 


230 


9.9 


26-30 


2,936 


2,239 


76.3 


2,093 


234 


11.2 


ol -ob 


1,641 


1,136 


69.2 


1,052 


111 


10.6 


ob-40 


798 


563 


70.6 


515 


58 


11.3 


Over 40 


593 


432 


72.8 


406 


21 


5.2 


Gender 














Male 


971 


442 


45.5 


406 


68 


16.7 


Female 


9,323 


7,545 


80.9 


7,002 


662 


9.5 


English is primary language 












No 


2,753 


2,093 


76.0 


1,920 


208 


10.8 


Yes 


7,728 


6,058 


78.4 


5,633 


526 


9.3 


Ethnic background 














Asian 


312 


215 


68.9 


188 


12 


6.4 


African American 


3,313 


2,617 


79.0 


2,346 


198 


8.4 


Hispanic 


2,552 


1,947 


76.3 


1,838 


204 


11.1 


Native American 


648 


540 


83.3 


513 


68 


13.3 


White 


3,621 


2,781 


76.8 


2,664 


257 


9.6 


Attained GED while in Even Start 












No 


7,956 


6,380 


80.2 


6,044 


534 


8.8 


Yes 


532 


427 


80.3 


418 


47 


11.2 


TOTAL 


10,683 


8,302 


77.1 


7,698 


751 


9.8 


Exhibit reads: At intake, nearly three quarters (72.1 percent) of Even Start adults targeted for the 
evaluation were not employed. Of those who were no employed, one in ten (9.8 percent) found part-time 
or full-time employment while participating in Even Start. 
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Section Twelve 

Cost of Even Start 



Even Start's budget has grown steadily since its inception, from $14.5 million in 1989 
to an estimated $90 million in 1 993. As Exhibit 1 .1 showed, this increase has enabled 
Even Start to fund more projects each year, so that well over 300 projects will be funded 
in 1993. Data on federal costs and number of families served are available for all Even 
Start projects, while projects in the In-Depth Study portion of the evaluation participated 
in a special assessment of the federal and local costs of Even Start. This study was 
conducted in the projects' second year of operation. The In-Depth Study projects were 
not randomly selected, but they do represent a reasonable cross-section of fully 
functioning Even Start grantees. 



Cost Per Participating Family 

Even Start projects engage in many functions including project administration and 
coordination, three core services, a range of support services, evaluation activities, 
recruiting, case management, and many others. One aim of this evaluation is to calculate 
the federal (Even Start) cost per family; a second purpose is to ascertain the full cost of 
all Even Start services, including federal Even Start funding, local matching funds, in-kind 
services or facilities, the value of other locally obtained core and support services, and 
other federal funding (e.g., the pro-rated cost of early childhood education obtained from 
Head Start or the pro-rated cost of adult basic education obtained from a local community 
college). 

Exhibit 12.1 shows the federal cost for Even Start projects participating in the national 
evaluation for the first three years of program operations. The number of families and 
participants (adults and children) served increased greatly from 1989-90 to 1990-91 , as 
would be expected given the increase both in number of projects funded and total federal 
dollars spent on the program. The number of projects in the evaluation remained constant 
from 1990-91 to 1991-92; however, the number of families and participants increased 
substantially once again (by 46 percent), indicating that projects were maturing and 
becoming more efficient. 
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Federal Cost for Even Start Projects, 
by Program Year 



Measure 


1989-90 


1990-91 


1991-92 


Number of Projects 


76 


123 


123 


Total Grant Awards (millions) 


$14. 5 


$24.2 


$24.2 


Total Families Served 


2,460 


6,596 


9,668 


Total Participants Served 


5,672 


16,143 


24,799 


Federal (Even Start) Cost Per 
Family 


$5,894 


$3,669 


$2,503 


Federal (Even Start) Cost Per 


$2,556 


$1,499 


$975 



Participant 



Exhibit reads: The federal (Even Start) cost for each Even Start family was $2,503 during the 1991- 
92 program year. 



The federal cost per Even Start family has declined over time, from $5,894 in 1989-90 
to $3,669 in 1990-91, and again to $2,503 in 1991-92 1 . The same pattern holds for 
the cost per participant-a decline from $2,556 in 1989-90 to $975 in 1991-92. 

Data from projects participating in the In-Depth Study cost analysis show that Even Start 
projects obtain substantial amounts of support, in addition to their federal Even Start 
funds, in order to deliver appropriate services to participating families. As noted above, 
the federal (Even Start) per family cost was $2,503 in 1 991 -92. The In-Depth Study cost 
analysis found that local projects obtained additional funding of $1 ,352 per family, for a 
total of $3,855 per family (see Exhibit 12.2). Thus, federal Even Start funds comprise 
65 percent ($2,503) of the total cost per family ($3,855), and other funds obtained 
locally (including local, state, and other federal) comprise 35 percent ($1 ,352) of the total 
cost per family. Alternatively, local projects are able to match 54 percent of their federal 
Even Start funds. 



'This calculation is based on total federal cost ? divided by the total number of families served and is appropriate 
for costing policy alternatives and making comparisons among federal programs. It also is possible to calculate 
the federal cost per family for a typical Even Start project-the mean cost per family is calculated for each project 
in the study and then an average of project means is computed. This approach gives equal weight to each project 
(rather than to each family), and yields a cost per family of $3,634 in 1991-92. The project-based cost per family 
is higher than the individual-based cost per family because large projects (which serve more families at a lower 
per-family cost) count no more than small projects (which serve fewer families at a higher per-family cost). This 
approach would be appropriate if the interest is in comparing costs across projects, or in determining which types 
of projects are most cost-efficient. 
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Exhibit 12*2: Federal Even Start Cost and All Other Costs 
(1991 -92 Program Year) 



Federal Even 
Start Costs 
(65 percent) 



\ 

$2,503 per family \ 
$975 per participant 




All Other Costs 
(35 percent) 



Total Costs - $3,855 per family; 

- $1 ,502 per participant 



Exhibit reads: Sixty-five percent of total Even Start costs is provided by federal Even Start funding; 35 percent 
is provided by other sources including local matching, in-kind services, or facilities, the value of other locally 
obtained core and support services, and other federal funding (e.g., the pro-rated cost of early childhood 
education obtained from Head Start). 



Distribution of Total Cost by Function 

More than half (55 percent) of all Even Start costs (federal and all other funds) were 
incurred in the provision of core services: 31 percent for early childhood education, 15 
percent for adult education, and 9 percent for parenting education (Exhibit 12.3). An 
additional 9 percent was spent on the provision of support services which are designed 
to enable families to participate in core service activities. Thus, almost two-thirds (64 
percent) of projects' funds were spent on the direct provision of services. Remaining 
funds were spent for program administration and coordination (14 percent), evaluation (10 
percent), case management and recruiting (4 percent), and for a variety of other functions 
(8 percent) such as field trips, staff meetings, clean-up, and errands. 
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Exhibit 12.3: Distribution of Even Start Costs (Both Federal and All Other) 
by Function (1991-92 Program Year) 



Early Childhood Education 




Evaluation 



Exhibit reads: 31 percent of Even Start costs {both federal and all other) were incurred in the provision of early 
childhood education services. 

Project Variation in Federal Cost Per Family 

As with most variables investigated in this evaluation, there is great variation across 
projects in the amount of federal dollars spent on each family. Exhibit 12.4 shows a 
distribution of project-level per family costs. 2 Some projects spend relatively few federal 
dollars per family while other projects spend much more on a per family basis. Most 
projects spend between $1,000 and $5,000 federal dollars per family. However, six 
projects spent less than $1,000 per family, while three projects spent over $8,000 per 
family. Such wide variation in expenditures per family reaffirms that Even Start projects 
have taken very different approaches to organizing and implementing services. It also 
suggests that there are wide project-to-project differences in access to federal and non- 
federal resources outside of Even Start. For example, some projects can use locally- 
available adult education and early childhood education services, while others must 
provide those services using their federal Even Start funds. 



^he federal cost per family at the project level is calculated by dividing the total number of families 
participating in core services during a year by the total federal funds received by the project for that year. 
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Exhibit 12.4: Project-Level Distribution of 
Federal Cost Per Family (1991-92 Program Year) 




1-2 2-3 3-4 4-5 5^ 6-7 7-« 

Federal Cost Per Family (Thousands of Dollars) 



Exhibit reads: Six projects spent less than $ 1 ,000 of their federal grant per family, while three projects spent 
over $8,000 per family. 

Information on two project characteristics that are related to cost per family is shown in 
Exhibit 1 2.5. One variable that is related to federal cost per family is the locus of primary 
responsibility for providing adult education and early childhood education services. As 
would be expected, projects which provide their own services expend more of their 
federal dollars on each family than projects that delegate responsibility for providing 
services to a cooperating agency. Eight projects retain primary responsibility for providing 
adult education and early childhood education. These projects spend a relatively large 
amount ($5,775) of their federal grant on each family because they are paying for the 
direct provision of services. Eighty-three projects share responsibility for providing core 
services with cooperating agencies. On average, these projects spend $3,579 of their 
federal dollars on each family. Finally, eight projects delegate primary responsibility for 
providing adult education and early childhood education to cooperating agencies. These 
projects spend an average of $1,878 of their federal dollars on each family. 

A second variable that relates to the amount of a project's federal grant spent on each 
family is the number of families served by the project. As would be expected, projects 
that serve large numbers of families spend fewer federal dollars per family, while projects 
that serve smaller numbers of families spend more federal dollars per family. 
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In 1991-92, there were 18 projects which served fewer than 30 families. On average, 
they spent $6,31 2 per family. The 66 projects that served between 30 and 99 families 
spent $3,742 per family. And finally, the 28 largest projects, which served 100 or more 
families did so at a federal cost of $1,659 per family. 



Exhibit 12.5 




Federal Cost Per Family by Project Size and 
Locus of Responsibility for Core Services 
(1991 92 Program Year) 


Project Characteristic 


Number of 
projects 


Average Grant 
Per Participating 
Family 


Locus of primary responsibility for adult education and early childhood education 
services 


Even Start 


8 


$5,775 


Both 


83 


$3,579 


Cooperating agency 


8 


$1,878 


Number of participating families 






Less than 30 families 


18 


$6,312 


30-99 families 


66 


$3,742 


100 or more families 


28 


$1,659 


Exhibit reads: Projects responsible for directly providing core services had higher costs than projects 
which relied on cooperating agencies. Small projects had higher costs than large projects. 


Note: Only projects reporting at least ten participating families were included in the analysis. 
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Section Thirteen 

Summary 



This section summarizes the findings contained in this report. The reader should realize 
that the analyses presented here will be expanded upon in the final report from this 
evaluation. 



Characteristics of Even Start Participants 

Based on data reported for the 1989-90, 1990-91. and 1991-92 program years. Even 
Start projects are serving the intended population. All of the participating Even Start 
households had at least one child between the ages of zero and seven, 77 percent of the 
adults who participated in Even Start core services did not complete high school, and 69 
percent of Even Start families had total annual income under $10,000. The Even Start 
population can be further described as follows: 

• 48 percent of Even Start families describe themselves as couples with 
children, 39 percent are single parent households, and 13 percent have 
extended families or other living arrangements. 

• 47 percent of Even Start families report job wages as their primary source 
of financial support, while 52 percent report that government assistance is 
their primary source of support. 

• Most adults in Even Start are between the ages of 22-29 (47 percent) or 
30-39 (31 percent). 

• 45 percent of Even Start adults are white, 27 percent are African-American, 
18 percent are Hispanic, 6 percent are Native American, and three perce it 
are Asian or Pacific Islander. 

• English is the primary language for 72 percent of Even Start adults; Spanish 
is the primary language for 22 percent. 

• Nine percent of the children served by Even Start were identified as having 
a disability. 
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Conclusions 



Implementation of Even Start Projects 



About $24 million was spent to fund 123 Even Start projects during the 1991-92 
program year. This was the third year of program operations for 76 of the projects, and 
the second year of operations for 47 projects. 



Core Services Delivered Through Even Start 

All Even Start projects are required to provide three "core services" for the education or 
instruction of adults and children: (1) parenting education services that assist parents in 
developing the capacity to function as teachers for their children, (2) adult education 
services that develop literacy skills of parents, and (3) early childhood education services 
that meet early education needs of children. As specified in the law, these core services 
can be provided either by staff funded through Even Start or by staff funded by 
cooperating agencies. In addition to the three core services, Even Start projects are 
required to provide educational and instructional services that involve parents and their 
children in joint activities, and to provide some of these activities in the home. 

Parenting Education Services. Even Start projects provided a wide range of services to 
help parents raise their children. For example, projects helped families make use of 
services provided by other social agencies, emphasized parents' role in the education of 
their children, oriented parents and children to school routines, furnished information 
about child development, trained parents in child behavior management, worked to build 
parental self-esteem, and instructed parents in life skills and in health and nutrition. Each 
of these different types of parenting education was provided by 90 percent or more of the 
Even Start projects. Depending on the specific service, 34 to 55 percent of the Even 
Start projects provided parenting education directly, 36 to 54 percent shared provision 
with a cooperating agency, and 5 to 10 percent delegated provision of parenting 
education to a cooperating agency. 

Adult Education Services. Almost 1 00 percent of the projects reported that they provided 
services to prepare adults to attain a GED, 86 percent provided services in adult 
education, and 92 percent provided services in adult secondary education. Instruction in 
English as a second language was provided by 59 percent of the projects. The locus of 
responsibility for providing adult education services differs from that of parenting 
education. About 30 percent of the projects provided adult education services directly 
by Even Start staff, another 30 percent of the projects provided services through 
cooperating agencies, and about 25 percent of the projects provided services jointly by 
Even Start and cooperating agency staff. 

Early Childhood Education Services. Children in Even Start projects were provided with 
a range of early childhood education services. Three different preschool options were 
used, with many projects using combinations: (1) 72 percent of the projects enrolled 
some of their children in Head Start, (2) 46 percent of the projects enrolled some of their 
children in a Chapter 1 -funded pre-K program, and (3) 93 percent of the projects provided 
some other preschool option. For children old enough to enter the public schools, most 
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Conclusions 



Even Start projects participated in joint planning activities with the public schools. Hence, 
85 percent of the projects included kindergarten as an Even Start service, and 74 percent 
of the projects provided early childhood education services to children under eight years 
of age who were in primary grades, again through the vehicle of joint planning with the 
public schools. 

As would be expected, all Head Start and Chapter 1 pre-K services were provided by 
cooperating agencies. About 38 percent of the "other preschool" services were provided 
directly by Even Start staff. This distribution is not surprising given the high cost of early 
childhood education services and their general availability through cooperating agencies 
and the public schools. 



Adult/Child Services Delivered Through Even Start 

Almost all of the Even Start projects (more than 90 percent) reported that they delivered 
a wide range of core services to parents and children together. This is an important part 
of the Even Start model in that it impresses on parents that they are a key to their child's 
education. Examples of services delivered to adults and children together include reading 
and story telling, developing readiness skills, social development and play, development 
of gross motor skills, working with numbers, arts/crafts, and health/nutrition. 



Support Services Delivered Through Even Start 

Even Start projects used funds to provide many different kinds of support services 
designed to enable families to participate in the core services. Support services remove 
barriers that, if unattended, restrict a family's ability to receive instructional and 
educational services. Several types of support services were provided by 85 percent or 
more of the projects including transportation, referrals for employment services, mental 
health services, family advocacy assistance with government agencies, child care, meals, 
counseling, nutrition assistance, and health care assistance. 

Across all types of support services. Even Start projects provided 22 percent directly, 
cooperating agencies provided 27 percent directly, and Even Start and cooperating 
agencies jointly provided another 28 percent. These findings suggest that Even Start 
projects did, as planned, obtain many existing support services from existing providers 
and stepped in to provide more immediate support services to enable families to 
participate in core services. 

Cooperative Arrangements 

Even Start projects are required to establish cooperative arrangements with other agencies 
to avoid duplicating services. This strategy allows optimal use of limited resources and 
allows projects to concentrate on providing new services to fill service gaps. 
Collaboration and cooperative arrangements were, indeed, a key focus of Even Start 
projects. During the 1991-92 program year. Even Start projects were involved in 2,808 
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cooperative arrangements to provide core services. This is an average of 27 cooperative 
arrangements per project. Fifty percent of the arrangements were for parenting 
education, 24 percent were for adult basic education, and 26 percent were for early 
childhood education. The most common cooperators were "other departments and 
programs within the public schools," "local, county, state or tribal agencies," and 
"postsecondary institutions." 



Implementation Problems 

Even Start projects reported several barriers to project implementation. The most 
common barriers were difficulties in the recruitment, retention, attendance, and 
motivation of families (41 projects), problems of communication and coordination with 
cooperating agencies (28 projects), financial problems {20 projects), staffing problems (16 
projects), problems with facilities and space (16 projects), and problems with the 
evaluation (15 projects). 



Participation in Even Start 

Number of Participating Families 

Participation in Even Start has been increasing over time. This is due both to the addition 
of new projects and to improved efficiency among existing projects. The number of 
families served by the 76 projects first funded in the fall of 1989 increased from 2,460 
in 1989-90 to 4,790 in 1990-91 and again to 5,567 in 1991-92. Thus, the same 
projects, with the same level of resources served more than twice as many families in 
their third year of operations as in their first year. 

This large increase over time in the number of families served can be attributed to the 
projects' ability to solve implementation problems that occur during the first year of 
program operations, for example, time had to be spent defining the program, recruiting 
staff, and setting up operations. Once these problems were solved projects became more 
efficient, with the extra time and resources being devoted to recruiting and serving 
additional families. 



Length of Participation 

Although Even Start projects are funded for four years, very few families take part for 
that amount of time. Of the families that began Even Start in 1989-90, 55 percent 
participated only in that first year, 25 percent participated in both the first and second 
program years, and 20 percent participated in three years. This pattern improved for 
families in projects that were first funded in 1990, where 28 percent of the families 
participated for one year, and 72 percent continued into a second year. 
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Reasons for Exiting 



There are many reasons for turnover of families in Even Start; some are indicators of 
program success, while others signal problem areas. Based on families that reported a 
reason for leaving, completion of the planned educational program was listed by 27 
percent of families that exited Even Start. Moving out of Even Start's catchment area 
was the most common reason for leaving the program, listed by 31 percent of families 
that exited. Fourteen percent of the families left because of a general lack of interest in 
the program. Another 14 percent reported that a family crisis of one sort or another 
prevented them from participating. Seven percent left the program because they became 
ineligible due to a change in the family situation, i.e., there was no longer an eligible child 
or adult in the family. Six percent gave a variety of reasons which suggested personal 
or structural conflicts or barriers to continued participation. These included medical 
reasons, work conflicts, pregnancy, scheduling conflicts, child care problems, and a lack 
of transportation. 



Participation in Core Services 

The Department of Education requires that all Even Start families participate in each of 
the three core service areas during their time in the program. Almost all families had a 
child that participated in early childhood education during each year of the study: 90 
percent in 1989-90, 37 percent in 1990-91, and 98 percent in 1991-92. Participation 
rates for parenting education also were quite high: 88 percent in 1989-90, 94 percent 
in 1990-91, and 93 percent in 1991-92. At the beginning of Even Start, participation 
rates were low for adult education; 54 percent in 1 989-90. The Department of Education 
and local projects worked hard to increase these rates to 79 percent in 1990-91 and to 
90 percent in 1991-92. The percentage of families participating in all three core services 
increased steadily over the three years of study, from 46 percent to 74 percent to 84 
percent. 



Amount of Core Services Received 

The typical family received core services in an average of between six and seven different 
months during their period of enrollment in Even Start. The total amount of service 
received by the typical family is 39 hours for adult education (mean of 86 hours), 27 
hours for parenting education (mean of 49 hours), and 84 hours for early childhood 
education (mean of 1 89 hours). There is great variation in these numbers across projects. 

On a monthly basis, the typical Even Start family received 8 hours of adult education 
(mean of 13 hours), 4 hours of parenting education (mean of 7 hours), and 14 hours of 
early childhood education (mean of 25 hours). There is a clear relationship between 
amount of early childhood education and age of child. The typical Even Start child less 
than one year of age received 4 hours per month. This amount increased to 5 hours per 
month for one-year-olds, 9 hours per month for two-year-olds, 14 hours per month for 
three-year-olds, and 22 hours per month for four-year-olds. 
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The Short-Term Effects of Even Start 



Implementation of an Even Start project should lead to an increase in the amount of 
services received by families in each core service area. Appropriate support services also 
should be provided. While information on the types and quantities of core services 
provided to Even Start participants was discussed above, a key issue for attributing any 
observed pretest-posttest gains to Even Start as opposed to other factors is the extent 
to which Even Start has been able to increase families' participation in the three core 
service areas over what it would have been in the absence of Even Start. 

Data from this study show that Even Start has substantially increased participation rates 
in the three core service areas. We estimate that without Even Start, 30 to 40 percent 
of the Even Start population would take part in adult education. This is substantially less 
than ;he 90 percent participation rate achieved for Even Start families during the 1 991-92 
program year. For parenting education, participation increases from eight percent without 
Even Start to 93 percent for families in Even Start. And for early childhood education, 
participation rates are estimated to be about 60 percent in the absence of Even Start and 
98 percent for families in Even Start. These increases in service levels give us confidence 
in attributing observed gains to Even Start rather than to extraneous factors such as 
normal maturation. 

Data to assess the short-term effects of Even Start are available from two sources: the 
In-Oepth Study which involved random assignment of families to participate in Even Start 
or in a control group, and the NEIS which provides national-level data on participating 
families. Short-term effects are measured in four areas: children, parenting literacy, 
parenting skills, and families. 

Exhibit 13.1 summarizes the short-term effects of Even Start as reported in this 
evaluation. It shows positive effects on many measures, especially when measured via 
the NEIS. The In-Depth Study, with its stronger research design, provides a more 
conservative assessment and shows fewer positive effects. What are we to make of the 
differences in findings between the NEIS and the Ir.-Depth Study? First, we should 
recognize that there are no contradictions in the two sets of results. That is, across all 
of the outcome measures, there are no cases where one data set showed that Even Start 
had a positive effect while the other data set showed that Even Start had a negative 
effect. This is important, because it indicates that the differences between the two sets 
of results are differences of intensity rather than differences in direction. 

Second, all things being equal, we would believe findings from the In-Deptn Study over 
findings from the NEIS. This is because the In-Depth Study has a stronger research 
design, including a randomly-assigned control group. However, not all other things are 
equal, and the In-Depth Study has weaknesses. It overre presents Hispanics, and it has 
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Summary of Even Start Effects 






Outcome Measure 


lUo 


iMEIS 


Children 






PreSchool Inventory 


+ 


+ 


reauoay ricture Vocabulary Test 


0 


+ 


uriHu b emergent Literacy lest 


0 


na 


Parents: Literacy levels 






Comprehensive Adult Student Assessment System 


0 


+ 


Attainment of a GED 


+ 


+ 


neaaing activities at home 


0 


na 


Writing activities at home 


0 


na 


Parents: Parenting and personal skills 






Personal skills 






Depression scale 


o 


na 


Self efficacy 


o 


i id 


Home learning environment 




Talking with child 




4_ 
T 


Play materials in home 


o 


J_ 

i 


Number of children's books in home 


n 


4_ 


Reading to child 


o 


4_ 

i 


Reading materials in home 


+ 


~j_ 


Learning activities at home 


o 


J_ 

i 


Teaching child 


r\ 
\J 




Family rules 


C\ 


i id 


Activities with child 


o 


1 lu 


Parent as a Teacher 


o 




Parent/child reading task 






Parent describes book 


o 


na 


Parent questions/responds to child 


o 


na 


Child responds 


o 


na 

i id 


Parent's general affect 


o 


na 


Child's general affect 


0 


na 


Parents' expectations and involvement 






Expectations for school success 


+ 


+ 


cxpeciaiions tor nign school graduation 


0 


+ 


Families 






Social support scale 


o 


na 


Family resources 




Source of income 


0 


na 


Income level 


0 


na 


Adequacy of resources 


0 


na 


Employment 


0 


0 



Notes: + signifies a positive program effect. 

o signifies no program effect. 

signifies a negative program effect, 
na means the measure was not included in this part of the study. 
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a relatively small sample size - 200 families from five projects. Thus, the statistical 
power of the In-Depth Study is substantially lower than the power of the NEIS. 

To sum up, the In-Depth Study and the NEIS data always point in the same direction, but 
in many cases the treatment/control group differences in the In-Depth Study are not 
statistically significant. If the In-Depth Study sample size was larger, then the same size 
treatment/control group differences would lead to more positive effects, as is seen in the 
NEIS. 

Even with the lack of power in the In-Depth Study, the overall picture is encouraging. 
Data from both the NEIS and the In-Depth Study show positive effects for children and 
for parents. There are no positive effects for families, but the outcomes (e.g., income, 
employment) in this area tend to be ones that Even Start does not expect to change in 
the short term. It will be important to determine whether the positive short-term effects 
for children and parents deteriorate over time or whether they are maintained, and 
whether it is possible to detect long-term effects for families. 



Effects on Children 

Effects on children were assessed using three measures: the PreSchool Inventory, the 
Peabody Picture Vocabulary Test, and the Child's Emergent Literacy Test. 

The PreSchool Inventory (PSI). The PSI is a 32-item individually-administered measure 
that assesses a range of school readiness skills, e.g., identifying shapes and colors, and 
understanding numerical concepts. It requires 1 5 minutes to administer and is appropriate 
for children between the ages of three and five. 

Because the PSI measures school readiness skills, children's scores on the PSI improve 
with age, simply as a function of maturation. This study found that children's PSI scores 
increase by .42 items per month due to normal maturation/development. However, by 
participating in Even Start, children's PSI scores increase at more than double the 
expected rate, by .92 items per month. This finding is based on data from the NEIS and 
is confirmed by data from the In-Depth Study which show that Even Start children gain 
significantly more than control group children on the PSI. 

This accelerated rate of learning on the PSI means that as Even Start children enter the 
public schools they are more likely to know basic concepts and precursors of kindergarten 
skills than they would have in the absence of the program. 

The Peabody Picture Vocabulary Test <PPVT). The PPVT measures receptive (hearing) 
vocabulary, and gives a quick estimate of verbal or literacy-related skills. An individually- 
administered test, it requires 15 to 20 minutes per child and is appropriate for children 
between the ages of two and 1 8. Whereas the PSI measures school readiness, the PPVT 
assesses verbal or language skills. 

Unlike the PSI, the PPVT has national norms. When administered the PPVT as a pretest, 
prior to entry into the program, Even Start children averaged 82 standard score points, 
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This corresponds to the twelfth percentile nationally and highlights the low verbal skills 
of children at entry to Even Start. 

Also unlike the PSI, the In-Depth Study and the NEIS tell different stories for the PPVT. 
Findings from the In-Depth Study show no significant program effect. On the other hand, 
data from the NEIS indicate that Even Start children gain more than would be expected 
without the program (0.83 standard score points per month while in Even Start vs an 
expected gain of 0.0 points per month). It is possible that Even Start is having a positive 
effect on the PPVT, that we are seeing that effect on the NEIS, but that we are not able 
to detect the effect in the In-Depth Study because of a combination of small sample size 
and overrepresentation of Hispanic children who, on average, do not gain as much on the 
PPVT as African-American children or white children. Additional data which may lead to 
firmer conclusions will be presented in the final report from this evaluation. 

Child's Emergent Literacy Test (CELT). The CELT was developed for this evaluation 
because no existing measure of children's emergent literacy skills was ideally suited to 
Even Start participants. The CELT includes items assessing orientation and directionality 
of text, recognition or letters and punctuation, purposes of reading, and children's 
knowledge and skills in writing their own name and age. 

The CELT was administered only in the In-Depth Study. While children in Even Start 
gained a statistically significant amount from pretest to posttest, children in the control 
group also gained a small amount, and the program effect <the gain of Even Start children 
over and above the gain of control group children) was not statistically significant. 



Effects on Parent Literacy 

This evaluation measured effects on parent literacy skills in three areas: (1) functional 
literacy levels on a reading test, (2) the percentage of Even Start adults who obtained 
their GED certificate, and (3) reading and writing activities in the home. 

The Comprehensive Adult Student Assessment System (CASAS). The CAS AS is an 
adult-oriented functional assessment system that measures a broad range of adult literacy 
skills and their application in real life domains including consumer economics, government 
and law, occupational knowledge, community resources, and health. This evaluation used 
only the CASAS reading survey. 

Even Start projects may have been effective in improving the functional literacy of 
participating adults. Data from the NEIS show that adults who participate in Even Start 
achieve small (3.6 scale score points), but positive gains on the CASAS reading survey, 
gains which are comparable in size to those observed in other studies of adult education 
programs. However, data from the In-Depth Study show that the gains of Even Start 
adults are not significantly greater than the gains achieved by a randomly-assigned control 
group. Both the NEIS and the In-Depth Study show that the amount of gain on the 
CASAS is directly related to the amount of instruction received through Even Start, and 
this finding strengthens our confidence that the observed gains can be attributed to the 
increase in adult education participation achieved by Even Start projects. The keys to 
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enhancing program effects in this area may be to strive for the fullest possible 
participation in adult education and for an increase in the number of hours per month. 

Attainment of a GED. Even Start projects promote attainment of a high school diploma 
or a high school equivalency diploma such as the GED, as do most adult education 
programs. The rationale for this goal is based upon a belief that a diploma or GED 
increases the chance the adult will find employment, provides opportunities for higher 
wages, and improves the quality of life through enhanced self-esteem. However, GED 
attainment is not an immediate or appropriate goal for all participants, nor is it emphasized 
by all Even Start projects. 

Even Start has a clear, positive effect on GED attainment, an effect which is supported 
by data from both the In-Depth Study and the NEIS. The NEIS shows that 7.1 percent 
of ail adults who entered Even Start without a GED or diploma achieved one while 
participating for a year or less in adult education services. The In-Depth Study supports 
the NEIS - it shows that 14.3 percent of participating adults in Even Start families 
attained a GED compared to 3.6 percent of participating adults in control group families. 

Compared to the total number of adults served, those that do attain a GED represent a 
relatively small percentage. One hypothesis is that a lack of retention and active 
participation in Even Start present major barriers to GED attainment. Most participants 
do not stay with the program long enough to overcome the lack of schooling, low English 
proficiency, poor test taking skills, or other reasons for not obtaining a diploma or GED. 
Some of the adults who had not attained a GED at the end of the reporting year for this 
analysis remain in Even Start and may, yet, achieve this goal. Data on this topic will be 
available for the evaluation's final report. 

Reading and Writing in the Home. Two sets of questions were included in the In-Depth 
Study to ask parents about their own reading and writing habits in the home. No 
measurable program effect was detected on the extent to which parents use reading and 
writing as literacy tools in the home. 



Effects on Parenting Skills 

In this evaluation, parenting skills were broadly defined to include parents' personal skills 
that may affect parenting, the activities and materials available in the home to foster 
children's development, parents' approach to reading to their children, and parents' 
expectations for their children. 

Personal Skills. Personal skills were assessed onl\ in the In-Depth Study. One measure 
of parents' personal skills is the CES-D scale of depressive symptoms. Scores above a 
certain level on the CES-D are an indication of clinical depression, and according to this 
criterion, 45 percent of Even Start parents in the In-Depth Study sites were clinically 
depressed at the pretest. These results are similar to those reported in other studies of 
low-income populations. However, we did not detect any effect of Even Start on the 
incidence of depressive symptoms. 
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A second measure of parents' personal skills is Pearlin and Schooler's Mastery Scale, a 
measure of self-efficacy or locus of control. As with depressive symptoms, we did not 
detect any effect of Even Start on parents' self-efficacy. 

Ho ne Learning Environment. Several scales describing different aspects of the home 
learning environment were included both in the In-Depth Study and in the NEIS. The In- 
Depth Study found positive program effects only on one of these scales: reading 
materials in the home. This scale measures the number of different types of reading 
materials found in thi home, e.g., books, magazines, newspapers. Although Even Start 
families showed positive gains on another scale (amount of play materials in the home), 
these gains were not significant given changes in the control group. No change was 
observed for several other scales including measures of learning activities in the home, 
teaching child at home, talking with child at home, family rules, activities outside home, 
and the parent as a teacher scale. 

On the other hand, data from the NEIS survey of all Even Start families show small but 
statistically significant gains on each measure of the home learning environment. The size 
of the gains are roughly what was seen for the In-Depth Study, however the NEIS data 
are based on a much larger sample which allows greater statistical power. Without a 
control group it is difficult to tell whether the observed gains are attributable to Even 
Start, and so the effects of Even Start on the home learning environment are mixed: we 
see clear, positive effects only on the amount of reading materials in the home, but there 
are indications that gains may be occurring in other areas. 

Parent-Child Reading Task. The In-Depth Study included a newly-developed measure of 
parent-child reading interactions. The parent was asked to read a simple book to her 
child, while a trained observer used a pre-coded rating form to record five major aspects 
of parent-child interactions: parent describes book to child, parent questions or responds 
to child, child responds, parent's general affect, and child's general affect. 

There was no effect of Even Start on any of the five areas of observation. While children 
did show a significant increase in their involvement and interaction during the reading 
activity (child's general affect) this increase wt^s not significant over and above the 
change seen for the control group. 

Parental Expectations. In both the In-Depth Study and the NEIS, parents were asked 
questions about how well they expected their child to do in high school and the likelihood 
that their child will graduate from high school. As is the case for such questions in most 
studies, parents generally had high expectations for their children at the pretest. Even so. 
Even Start significantly increased parental expectations as assessed in both the In-Depth 
Study and the NEIS. 



Effects on Families 

The In-Depth Study included several measures of the effects of Even Start on participating 
families, including their perceived social support, adequacy of financial resources, income 
level and sources and employment status. The NEIS instruments had questions about 
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income and employment. Across these measures, gains from program entry to the end 
of the first program year were minimal, It is possible that these areas are difficult to 
change because they are affected by circumstances beyond the control of the program, 
such as the local economy and the availability of jobs. It is also likely that these indices 
will not show large changes in the short-run, but instead require longer interventions and 
other positive short-term impacts in order to achieve significant gains. 

While there are few effects for families on these quantifiable variables, Even Start 
participants describe a number of qualitative changes in their lives and the lives of their 
children. Listening to the personal stories of program participants, it becomes apparent 
that most of the changes in attitudes and skills that the parents see in themselves and 
their children are positive short-term impacts of the program. Parents describe 
themselves as moving toward their goals of an educational certificate, getting a job and 
being a better parent. Given the current status of Even Start parents, it is reasonable that 
we do not yet see changes in the more distal outcomes of increased employment and 
income. 



Cost of Even Start 

The federal cost per Even Start family has declined over the life of the program, from 
$5,894 in 1989-90 to $3,669 in 1990-91 , and again to $2,503 in 1991-92. This is due 
to increases in the number of families served each year, indicating that over time, projects 
have matured and bee .me more efficient. 

Even Start projects also obtain substantial resources (e.g., matching funds, in-kind 
contributions, and the value of referred services), in addition to their federal Even Start 
funds, in order to deliver appropriate services to participating families. In 1991-92, 
$2,503 in federal funding per family was augmented by $1,352 in other resources to 
arrive at total resources of $3,855 per Even Start family. Thus, federal Even Start funds 
comprise 65 percent of the total resources per family and other funds comprise 35 
percent. 

Almost two-thirds (64 percent) of all Even Start resources were spent on the direct 
provision of services: 31 percent for early childhood education, 15 percent for adult 
education, 9 percent for parenting education, and 9 percent for support services. Even 
Start funds also were spent for program administration (14 percent), evaluation (10 
percent), case management and recruiting (4 percent), and for a variety of other functions 
(8 percent). 

Even Start costs vary tremendously across projects. As might be expected, projects 
which serve large numbers of families do so at a lower cost per family. In particular, 
projects which serve 100 or more families do so at a federal cost of $1,659 per family, 
while projects which serve 30 or fewer families spend an average of $6,31 2 per family. 
Also, projects which delegate responsibility for providing core services to cooperating 
agencies have a lower cost ($1,878 per family) than projects which retain primary 
responsibility for providing core services (cost of $5,775 per family). 
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